If while loop consumes the input from the pipe, it is launched in a subshell, thus all changes to the variables inside the loop do not work as expected (are not propagated to calling shell). Solution: loop should be executed by primary shell:

1. From Modifying variable inside while loop is not remembered:

   VAR=0
   while read file
   do
       [ "$file" = "tmp" ] && VAR=1
   done < <(ls -1)

   Process substitution which is used in above example, is described here. Another good application of process substitution is capturing the exit code of first command in a pipe:

   if mount /dev/sda /mnt 2> >(head -1)
   then
       ...
   fi

   The same way read and arrayread when applied after the pipe initialize the variable in a forked shell (hence echo file.txt | read line does not work) so the solution looks this:

   read file < <(echo file.txt)

2. From Create variables and assign values via loop:

   VAR=0
   for file in $(ls -1)
   do
       [ "$file" = "tmp" ] && VAR=1
   done

Intersection of two lists:

• cat 1.txt 2.txt | sort | uniq -d
• comm -12 1.txt 2.txt

Symmetric difference of two lists:

• cat 1.txt 2.txt | sort | uniq -u

• ( >&2 echo "bla-bla" );
• echo "bla-bla" > /dev/stderr
• ... | tee /dev/stderr | ...

From How to assign a heredoc value to a variable in Bash:

read -r -d '' SCRIPT <<'EOS'
use strict;
 
my $lines = 0;
 
while (<>) {
    chomp;
 
    $files++;
}
 
print "Number of lines: $lines\n";
 
EOS
 
perl -e "$SCRIPT"

... | tail -n +2

From Use bash to read line by line and keep space and Bash read line does not read leading spaces:

while IFS= read -r line
do
    echo "$line"
done < some_file.txt

Similarly to read the whole file into variable:

IFS= FILE=`cat some_file`

(
echo -e "File\tMD5\tSize\tCTime"
find . -type f -printf '%p\t%s\t%T+\n' | sort -k1 -t$'\t' | while IFS= read -r line
do
     echo -e "$(md5sum "`echo "$line" | cut -f1`" | cut -f1 -d' ')\t$line"; \
done
) | column -t -s $'\t'

See also:

• How can I use column to delimit on tabs and not spaces?, How to define 'tab' delimiter for 'cut'?
• Quoting within $(command substitution) in Bash

How can I use xargs to copy files that have spaces and quotes in their names actually says that xargs should be tweaked, but if the list is produced by find then solutions are:

• Use null separator: find ... -print0 | xargs -0 wc -l
• Use find ability to append arguments: find ... -exec wc -l {} +

seq 1 10 | paste -s -d+ | bc

$ cat file | tr -d -c '#' | wc -c

See Advanced Bash-Scripting Guide, Chapter 7:

if [[ $fullpath == *.txt ]]
then
    echo "This is text file"
fi

From Extract filename and extension in bash:

file=`basename "$fullpath"`
extension=${file##*.}
filename=${file%.*}

array=( "${array[@]/#/prefix_}" )
array=( "${array[@]/%/_suffix}" )

To print an array as new-line separated string:

IFS=$'\n' echo "${array[*]}"
IFS=$'\n' echo "$*" | xargs ...

To generate e.g. 5 random passwords each having 16 characters in alphabet “A-Za-z0-9” use:

tr -dc A-Za-z0-9 < /dev/urandom | fold -w 16 | head -5

$ myprog
# press Ctrl+Z to suspend the program
$ bg
$ disown
# switch to another terminal / screen / tmux session
$ reptyr $(pgrep myprog)

To disable bash from sending bell to terminal add/uncomment the following in ~/.bashrc:

set bell-style none

However bell can be useful to notify long running task:

• Configure PuTTY to flash Windows taskbar on bell
• Latest ConEmu (v170807) supports this out-of-box unless Disable all flashing is not enabled in Settings → Features.

The problem is solved using stdbuf utility:

stdbuf -oL tail -f /var/log/messages | tee logs.txt