3

Watching a directory for changes

 1 year ago
source link: https://willschenk.com/labnotes/2023/watching_a_directory_for_changes/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

fswatch is a cross platform command that will let you watch a directory for changes. Lets give it a try:

  brew install fswatch

And we can test it

  mkdir
  fsevent tmp

And then in another terminal:

  touch tmp/b

And see if the first one prints out anything!

Docker

Ok so that's simple. Lets see if we can put it in a Docker container and see if it works across volumes

  from debian:12

  run apt-get -q update && apt-get install -y fswatch

  cmd fswatch /data

Then build

  docker build . -t fswatch_test

Finally, start it up to test:

docker run --rm -it -v ./tmp:/data fswatch_test

Filtering on events

Events are a funny name here, because we can use it both to look at the Created and Updated event, but also to filter out on file types like IsFile or IsDirectory.

  fswatch -x --event Updated --event IsFile tmp

Simple job processing queue

Lets have two scripts, one that writes files out into a directory when recording events, and another that loads them up as they come in and does something with them.

job_add:

  #!/bin/bash

  directory=queue
  mkdir -p ${directory}
  if [ -z "$1" ]; then
      command=event=ping
  else
      command=${@}
  fi

  time=$(date +"%Y-%m-%d-%H:%M:%S")

  outfile=${directory}/${time}.job

  jo ${command} > ${outfile}
  cat ${outfile}

We can take a look at this and some various outputs:

bash job_add
{"event":"ping"}
bash job_add event=build id=1234
{"event":"build","id":1234}

Watch for new files

Now we can implement job_watcher. This first looks into the queue directory for all of the job files, and if it don't have a log file then calls process_job. After that, it starts up fswatch and runs each file as it changes.

  #!/bin/bash

  function setup {
      directory=queue
      mkdir -p ${directory}
  }

  # Watch for file changes
  function watch_for_changes {
      fswatch --event Updated \
              --event IsFile \
              ${directory} | \
          while read line ;
          do
              # Only react to .job files
              if [[ $line =~ ".job" ]]; then
                  if [ -f $line ]; then
                      process_job $line
                  fi
              fi
          done
  }

  # Look for all jobs that haven't been run yet
  function catch_up {
      ls -1 ${directory}/*job | \
          while read job ;
          do
            outfile=$(echo $job | sed s/\.job/\.log/)

            if [ ! -f $outfile ]; then
                echo Running $job
                process_job $job
            fi
          done
  }

  function process_job {
      type=$(cat $1 | jq -r '.event')
      outfile=$(echo $1 | sed s/\.job/\.log/)

      if [ $type == 'ping' ]; then
          echo pong > $outfile
          echo Got ping event
      else
          echo error > $outfile
          echo Unknown event $type
      fi

  }

  setup
  catch_up
  watch_for_changes

Dockerfy

And lets see if we can communicate across the containers! With a new Dockerfile.watcher file:

  from debian:12

  run apt-get -q update && apt-get install -y fswatch jo jq git

  workdir /app
  copy job_watcher .

  cmd bash /app/job_watcher

Easy build with:

  docker build . -f Dockerfile.watcher -t watcher_test
  docker run --rm -it -v ./queue:/app/queue watcher_test

Then if we add a couple of jobs

  bash job_add
  bash job_add event=test

We'll get in return:

Got ping event
Unknown event test

References


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK