English | 日本語(Japanese)

rf is a CLI tool that filters text/JSON/YAML with Ruby code. These files are processed using tools such as awk, jq, and yq, each of which requires you to learn how to use them and their syntax. With rf, you can write equivalent processing in Ruby code, making it a tool that Rubyists can easily use.

rf is built using mruby and consists of a single binary file. You don't need to install Ruby separately to use rf. One of the features of rf is that you can start using it immediately by downloading the binary file.

Replace part of the text

rf 'gsub(/hello/, "world")' example.txt

Read from standard input

rf 'gsub(/hello/, "world")' < example.txt

Display only lines that contain specific patterns

rf '/hello/' example.txt

UTF-8 can also be used for specified patterns

rf '/🍣/' example.txt

Add the second column of comma-separated files

rf -F, 's||=0;s+=_2; at_exit{puts s}' example.csv

Output a specific key in a JSON file

rf json '.a.b' example.json

Output a specific key in a YAML file

rf yaml '.a.b' example.yaml

You can download rf from the Release page. Binary files are provided for each platform and architecture. Please refer to the following table for each corresponding status.

You can also install it using each package manager as another method.

asdf plugin install rf https://github.com/buty4649/asdf-rf/
asdf install rf latest
asdf global rf latest

rtx plugin install rf https://github.com/buty4649/asdf-rf/
rtx install rf@latest
rtx global rf@latest

brew tap buty4649/tap
brew install buty4649/tap/rf

# if not installed k1LoW/gh-setup
gh extension install k1LoW/gh-setup
gh setup --repo buty4649/rf --bin-dir ~/.local/bin

Usage:
  rf [command] [options] 'command' file ...
  rf [command] [options] -e program_file file ...

Commands:
  text       use Text filter (default)
  json       use JSON filter
  yaml       use YAML filter
  version    show version
  help       show this message

Options:
  -F, --filed-separator string
      --[no-]color                  [no] colorized output (default: --color in TTY)
  -g, --grep                        Interpret command as a regex pattern for searching (like grep)
      --include-filename string     searches for files matching a regex pattern
  -i, --in-place[=SUFFIX]           edit files in place (makes backup if SUFFIX supplied)
  -R, --recursive                   read all files under each directory recursively
  -f, --file string                 executed the contents of program_file
  -s, --slurp                       read all reacords into an array
  -q, --quiet                       suppress automatic printing
  -H, --with-filename               print filename with output lines
      --with-record-number          print record number with output lines
  -h, --help                        show this message and exit

If you don't specify file, it will be read from standard input.

Write Ruby code in command. In rf, this Ruby code is called command. rf uses mruby as a Ruby processing system. mruby is a lightweight Ruby processing system designed for embedded systems. It is compatible with CRuby and can use almost all functions, but there are some function restrictions. For more information, please refer to mruby documentation.

In rf, processing is performed in the following way.

1. Read data from file or standard input.
2. Convert the read data into a format that is easy to handle with Ruby.
3. Divide the converted data.
4. Execute command and evaluate the result.
5. Process the evaluation result and output it.
6. Repeat steps 3 through 5 until there is no more data.

The functions that perform steps 2, 3 and 5 are called filters in rf. If you express this process flow in pseudo-code of Ruby, it will be as follows.

Filter.read(input).each do |record|
  puts eval(command)
end

Filters divide input data according to certain rules. In rf, this divided unit is called record. The rules for dividing vary depending on the filter.

In text filters, they are divided by newline codes (\n). In JSON filters/YAML filters, they are divided by object units. For example, if the input data is an array, it will be divided into each element of the array. If expressed in Ruby code, it will be processed as follows.

# In case of YAML filter it will be YAML.load
[JSON.load(input)].flatten(1).each do |record|
  puts eval(command)
end

In rf, the part that is further divided from the record is called field. By using the special variable _1, _2, _3, etc., you can access the first field, second field, third field, etc. The field division is done automatically, but you can access records that are not divided into fields by using the _ variable.

The field division rule varies depending on the filter. In the case of a text filter, it is separated by whitespace characters. It behaves the same as when String#split is called without arguments. By specifying the -F option, you can change the delimiter. It is also possible to use regular expressions as delimiters. JSON filters/YAML filters only divide into fields if the record is an array. If it is another data type, only _1 will be stored and _2 and later will be nil.

In rf, you can use special variables defined in Ruby. However, some variables have been changed to rf's own definitions.

Several methods are built-in for convenience.

rf extends Ruby's language features.

In standard Ruby, String and Integer/Float cannot be calculated directly. You need to call String#to_i or String#to_f. rf automatically performs these calls.

1 + "1"
#=> 2

You can call a key in a Hash as a method.

h = {"a": 1}
h.a
#=> 1

If you call an undefined key as a method, it will become undefined method.

h.b
#=> Error: undefined method 'b'

You cannot call keys that contain whitespace characters, symbols or multibyte characters in this way. Use Hash#[].

See LICENSE © buty4649