How to Run Multiple Commands Simultaneously in Linux

You may know the how to use & to run commands in the background, but it also has this other very interesting use case.

We can run two or more commands asynchronously using the single ampersand & character.

Asynchronous Commands

command1 & command2 & command3 ...

Suppose we had three scripts we wanted to run simultaneously.

./s1.py & ./s2.py & ./s3.py

All three Python scripts will run simultaneously in separate sub-shells, allowing you to take advantage of multiple cores. The output of all three, however, will all be attached to this parent shell. This means each line will be displayed on a first-come, first-serve basis.

If you’re having trouble running Python scripts through bash like this, you may have forgotten to add #!/usr/bin/env python to the top of your script. Another alternative is just to run the scripts through the python command python s1.py.

If you need to examine the output of each separately, it may be helpful to redirect each script output to a file.

./s1.py > log1.txt & ./s2.py > log2.txt & ./s3.py > log3.txt

Because this is asynchronous, none of these commands should depend on one another.

For example, this string of commands would cause issues.

./s1.py > out1.txt & grep "some_text" out1.txt

In the second command, we are looking for the text some_text in the output of the first command, which doesn’t exist yet.

This is why we may need synchronous commands.

We can run two or more commands synchronously using double ampersands &&.

Synchronous Commands

command1 && command2 && command3 ...

One thing to note with && is that each subsequent command is dependent on the success of the previous command.

If command1 fails, command2 will never run. If command2 fails, command3 will never run, and so on .

Continuing with our previous example:

./s1.py > out1.txt && grep "some_text" out1.txt

This is a valid command using &&.

grep "some_text" out1.txt will run as long as ./s1.py doesn’t contain any errors.