xtee is a command-line utility for reading from standard input and writing to both standard output and file, providing powerful features for file manipulation and data processing. It combines the functionality of tee, the soaking behavior of sponge, and more.

• Features
• Installation
  • Install release binaries (Without Go Installed)
  • Install source (With Go Installed)
    • go install ...
    • go build ... the development Version
• Usage
  • Examples
    • Basic
    • Deduplicating Files
    • In-Place Deduplicating Files
    • Appending Lines to File
    • Appending Unique Lines to File
  • Performance Considerations
• Contributing
• Licensing

• Supports streams spliting, incoming standard input into standard output and file
• Supports soaking up input before writing, enabling safe in-place processing
• Supports appending and overwriting the destination entirely
• Supports deduplication, keeping destination unique
• Cross-Platform (Windows, Linux & macOS)

Visit the releases page and find the appropriate archive for your operating system and architecture. Download the archive from your browser or copy its URL and retrieve it with wget or curl:

• ...with wget:

   wget https://github.com/hueristiq/xtee/releases/download/v<version>/xtee-<version>-linux-amd64.tar.gz

• ...or, with curl:

   curl -OL https://github.com/hueristiq/xtee/releases/download/v<version>/xtee-<version>-linux-amd64.tar.gz

...then, extract the binary:

tar xf xtee-<version>-linux-amd64.tar.gz

curl -sL https://github.com/hueristiq/xtee/releases/download/v<version>/xtee-<version>-linux-amd64.tar.gz | tar -xzv

...move the xtee binary to somewhere in your PATH. For example, on GNU/Linux and OS X systems:

sudo mv xtee /usr/local/bin/

Before you install from source, you need to make sure that Go is installed on your system. You can install Go by following the official instructions for your operating system. For this, we will assume that Go is already installed.

go install -v github.com/hueristiq/xtee/cmd/xtee@latest

• Clone the repository

   git clone https://github.com/hueristiq/xtee.git 

• Build the utility

   cd xtee/cmd/xtee && \
   go build .

• Move the xtee binary to somewhere in your PATH. For example, on GNU/Linux and OS X systems:

   sudo mv xtee /usr/local/bin/

  Windows users can follow How to: Add Tool Locations to the PATH Environment Variable in order to add xtee to their PATH.

To display help message for xtee use the -h flag:

xtee -h

help message:

      _
__  _| |_ ___  ___
\ \/ / __/ _ \/ _ \
 >  <| ||  __/  __/
/_/\_\\__\___|\___|
             v0.0.0

USAGE:
 xtee [OPTION]... <FILE>

INPUT:
     --soak bool          buffer input before processing

OUTPUT:
     --append bool        append lines
     --unique bool        unique lines
     --preview bool       preview lines
 -q, --quiet bool         suppress stdout
 -m, --monochrome bool    stdout in monochrome
 -s, --silent bool        stdout in silent mode
 -v, --verbose bool       stdout in verbose mode

echo "zero" | xtee stuff.txt

cat stuff.txt

zero

cat duplicates.txt

zero
one
two
three
zero
four
five
five

cat duplicates.txt | xtee --unique deduplicated.txt

zero
one
two
three
four
five

cat deduplicated.txt

zero
one
two
three
four
five

cat stuff.txt

zero
one
two
three
zero
four
five
five

cat stuff.txt | xtee stuff.txt --soak --unique

zero
one
two
three
four
five

cat stuff.txt

zero
one
two
three
four
five

cat stuff.txt

one
two
three

cat new-stuff.txt

zero
one
two
three
four
five

cat new-stuff.txt | xtee stuff.txt --append 

zero
one
two
three
four
five

cat stuff.txt

one
two
three
zero
one
two
three
four
five

cat stuff.txt

one
two
three

cat new-stuff.txt

zero
one
two
three
four
five

cat new-stuff.txt | xtee stuff.txt --append --unique

zero
four
five

cat stuff.txt

one
two
three
zero
four
five

cat new-stuff.txt | xtee stuff.txt --append --unique > added-lines.txt

cat added-lines.txt

zero
four
five

• For large files (>1GB), --soak mode will use significant memory as it buffers everything
• --unique mode maintains all seen lines in memory - be cautious with huge datasets
• Stream processing (without --soak) is most memory-efficient for large inputs

Contributions are welcome and encouraged! Feel free to submit Pull Requests or report Issues. For more details, check out the contribution guidelines.

A big thank you to all the contributors for your ongoing support!

This package is licensed under the MIT license. You are free to use, modify, and distribute it, as long as you follow the terms of the license. You can find the full license text in the repository - Full MIT license text.