2

Oil 0.9.4 - User Feedback

 2 years ago
source link: http://www.oilshell.org/blog/2021/11/release-0.9.4.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client
User Feedback

blog | oilshell.org

Oil 0.9.4 - User Feedback

2021-11-20

This is the latest version of Oil, a Unix shell that's our upgrade path from bash:

Oil version 0.9.4 - Source tarballs and documentation.

To build and run it, follow the instructions in INSTALL.txt. The wiki describes How To Test OSH and Where To Send Feedback.

If you're new to the project, see Why Create a New Shell? and posts tagged #FAQ.

What's Happened Recently?

Many things have happened in the project, but as usual the blog is behind! Highlights:

  • We had the first "oil-dev walkthrough" over videoconference yesterday. It surfaced a lot of great feedback and context!
  • It was motivated in part by a proposal to use Oil in Nix, and the resulting discussion.

I also didn't forget about the "big ideas" I wrote about this summer. I plan to return to those in some form.

However, this is a "workmanlike" release, to get back in the rhythm of things after traveling and moving.

Summary of Changes

For the last couple weeks, I focused on fixing user-reported bugs and addressing feature requests. Thanks to Miles Alan and bb010g for particularly thorough testing.

Prompt Enhancements

To customize your bash prompt, you use a cryptic language in the $PS1 variable. This release implements more parts of that language:

  • \s and \v evaluate to the shell and version -- e.g. osh and 0.9.4
  • \A and \D{} for substituting the current time -- 24 hour format and strftime() format, respectively

We also ensure that $PS1 is set in interactive shells. (An interactive shell is one that's started without a script argument, and where stdin is connected to a terminal.)

The read builtin

  • If the read() syscall is interrupted, run trap handlers and retry. (bash and zsh agree on this behavior; it's not clear what POSIX says.)
  • Set the terminal mode appropriately for read -d vs. read. This fixes a bug where the backspace key didn't work interactively.

Bug: Pipelines Aren't Blocked on Background Processes

Miles Alan reported an interesting process concurrency bug. It dates back to late 2018 when pipelines started to use the more sophisticated shopt -s lastpipe semantics.

By default, OSH runs the last element of a pipeline in the main shell process. Among other things, this means that the variable binding in echo foo | read myvar persists.

The bug is too subtle to explain here, but it reminds of this canonical example of why process-based concurrency is tricky:

sleep 1 &  # start background process

# The shell has to wait() for 2 processes in this pipeline
my-command | wc -l

But the sleep process might finish before both, in between them, or after both.

In other words, process done notifications via waitpid(-1) are asynchronous, and serialized, so shells need data structures to keep track of it all. This is an important part of the #shell-runtime I haven't written about.

The Oil Language

The feedback above is solidly in the OSH portion of the project, but we use it to improve the Oil language. Some thoughts:

  • Oil still needs a prompt hook, but we don't want to use the $PS1 language. It interleaves parsing and evaluation in a confusing and potentially unsafe way.
    • Instead, Oil will likely use the builtin sub mechanism with procs. If you're interested in this, let me know on issue 498!
  • The read changes reminded me that the read invokes the read() syscall for every byte! This is very unbuffered I/O!
    • So I've defined Oil's read --line and read --all to use buffered I/O instead. I think this is more appropriate for batch programs, and is consistent with Python and Perl.
  • The pipeline fix above applies to both OSH and Oil. In both cases, pipelines behave like zsh: they behave as if the bash shopt -s lastpipe option is set.
    • Recall that OSH and Oil are variants of the same interpreter, and live in the same binary. In particular, they share the runtime that executes pipelines. We speak of them separately, but there's a smooth upgrade path from one to another.

Closed Issues

#1002 Pipeline launched from main process waits for a background process

#1001 Unhandled termios exception

#1000 read builtin: backspace interpreted as ^?

#996 osh throws an `OverflowError` & dumps a stack trace on `(return 2147483648)` or `(exit 2147483648)`

#986 Builtin read not interruptible via signal

#982 Set PS1 in interactive shells

#962 Support \A in PS1

#748 Support \D{} in PS1 and PS0

What's Next?

I want to write these blog posts next:

Oil 0.9.3 - Extended Globs for Nix. Motivated by Nix, I implemented one of the "last" OSH features. This is a good opportunity to restate and explain the principles behind the design and implementation of OSH. I hope to answer many questions on the RFC thread with concrete examples and general principles.

Oil Winter Blog Backlog. As mentioned, the blog is backed up, and I want to catch up. This might overflow into a few parts, like the Summer Blog Backlog did (part 1 and part 2).

Links

  • In case I don't write about it soon enough on the blog, here's a Hacker News comment that summarizes the state of the project.
  • There is also more "content" on the #blog-ideas Zulip channel. Feel free to reply there and let me know what you want to read!

Reminder: Please try Oil and send feedback! The best feedback results from trying OSH and Oil on real programs. I just got some great feedback from Arturo Martín-de-Nicolás, which surfaced at least one bug, and reminded me of a gap in the Oil language (issue #1011).

Appendix: Metrics for the 0.9.4 Release

These metrics help me keep track of the project. Let's compare this release with version 0.8.12, which I announced in July.

Spec Tests

The spec test suites for both OSH and Oil continue to expand and turn green. The 0.9.3 work on extended globs resulted in whole suite of tests, because they can appear in many contexts.

Source Code Size

We have ~250 significant lines of code:

And ~600 new lines of physical code:

Benchmarks

I started using new benchmark machines in version 0.9.2, so let's use that as our baseline. It looks like there was no change.

Runtime:

Native Code Metrics

I've been maintaining the C++ translation behind the scenes, trying to make sure we don't dig ourselves into a hole. This comparison looks OK:

But I noticed a regression versus version 0.9.3, released last month:

Looking over the results, I see many 255 exit codes that are supposed to be 0. I believe this is due a change that started truncating exit codes, which fixed a user-reported crash. I suspect it didn't translate to C++ correctly, e.g. because Python has arbitrary size integers and C++ has fixed size integers.

We have ~1000 more lines of native code:

But curiously the binary is slightly smaller:


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK