How can I exclude directories from grep -R?

Question

I want to traverse all subdirectories, except the "node_modules" directory.

Answer 1

SOLUTION 1 (combine find and grep)

The purpose of this solution is not to deal with grep performance but to show a portable solution : should also work with busybox or GNU version older than 2.5.

Use find, for excluding directories foo and bar :

find /dir \( -name foo -prune \) -o \( -name bar -prune \) -o -name "*.sh" -print

Then combine find and the non-recursive use of grep, as a portable solution :

find /dir \( -name node_modules -prune \) -o -name "*.sh" -exec grep --color -Hn "your text to find" {} 2>/dev/null \;

SOLUTION 2 (using the --exclude-dir option of grep):

You know this solution already, but I add it since it's the most recent and efficient solution. Note this is a less portable solution but more human-readable.

grep -R --exclude-dir=node_modules 'some pattern' /path/to/search

To exclude multiple directories, use --exclude-dir as:

--exclude-dir={node_modules,dir1,dir2,dir3}

SOLUTION 3 (Ag)

If you frequently search through code, Ag (The Silver Searcher) is a much faster alternative to grep, that's customized for searching code. For instance, it automatically ignores files and directories listed in .gitignore, so you don't have to keep passing the same cumbersome exclude options to grep or find.

Answer 2

Recent versions of GNU Grep (>= 2.5.2) provide:

--exclude-dir=dir

which excludes directories matching the pattern dir from recursive directory searches.

So you can do:

grep -R --exclude-dir=node_modules 'some pattern' /path/to/search

For a bit more information regarding syntax and usage see

• The GNU man page for File and Directory Selection
• A related StackOverflow answer Use grep --exclude/--include syntax to not grep through certain files

For older GNU Greps and POSIX Grep, use find as suggested in other answers.

Or just use ack (Edit: or The Silver Searcher) and be done with it!

Answer 3

If you want to exclude multiple directories:

"r" for recursive, "l" to print only names of files containing matches and "i" to ignore case distinctions :

grep -rli --exclude-dir={dir1,dir2,dir3} keyword /path/to/search

Example : I want to find files that contain the word 'hello'. I want to search in all my linux directories except proc directory, boot directory, sys directory and root directory :

grep -rli --exclude-dir={proc,boot,root,sys} hello /

Note : The example above needs to be root

Note 2 (according to @skplunkerin) : do not add spaces after the commas in {dir1,dir2,dir3}

Answer 4

This syntax

--exclude-dir={dir1,dir2}

is expanded by the shell (e.g. Bash), not by grep, into this:

--exclude-dir=dir1 --exclude-dir=dir2

Quoting will prevent the shell from expanding it, so this won't work:

--exclude-dir='{dir1,dir2}'    <-- this won't work

The patterns used with --exclude-dir are the same kind of patterns described in the man page for the --exclude option:

--exclude=GLOB
    Skip files whose base name matches GLOB (using wildcard matching).
    A file-name glob can use *, ?, and [...]  as wildcards, and \ to
    quote a wildcard or backslash character literally.

The shell will generally try to expand such a pattern itself, so to avoid this, you should quote it:

--exclude-dir='dir?'

You can use the curly braces and quoted exclude patterns together like this:

--exclude-dir={'dir?','dir??'}

Answer 5

If you are grepping for code in a git repository and node_modules is in your .gitignore, you can use git grep. git grep searches the tracked files in the working tree, ignoring everything from .gitignore

git grep "STUFF"

Answer 6

Frequently use this:

grep can be used in conjunction with -r (recursive), i (ignore case) and -o (prints only matching part of lines). To exclude files use --exclude and to exclude directories use --exclude-dir.

Putting it together you end up with something like:

grep -rio --exclude={filenames comma separated} \
--exclude-dir={directory names comma separated} <search term> <location>

Describing it makes it sound far more complicated than it actually is. Easier to illustrate with a simple example.

Example:

Suppose I am searching for current project for all places where I explicitly set the string value debugger during a debugging session, and now wish to review / remove.

I write a script called findDebugger.sh and use grep to find all occurrences. However:

For file exclusions - I wish to ensure that .eslintrc is ignored (this actually has a linting rule about debugger so should be excluded). Likewise, I don't want my own script to be referenced in any results.

For directory exclusions - I wish to exclude node_modules as it contains lots of libraries that do reference debugger and I am not interested in those results. Also I just wish to omit .idea and .git hidden directories because I don't care about those search locations either, and wish to keep the search performant.

So here is the result - I create a script called findDebugger.sh with:

#!/usr/bin/env bash
grep -rio --exclude={.eslintrc,findDebugger.sh} \
--exclude-dir={node_modules,.idea,.git} debugger .

Answer 7

You could try something like grep -R search . | grep -v '^node_modules/.*'

Answer 8

Many correct answers have been given here, but I'm adding this one to emphasize one point which caused some rushed attempts to fail before: exclude-dir takes a pattern, not a path to a directory.

Say your search is:

grep -r myobject

And you notice that your output is cluttered with results from the src/other/objects-folder. This command will not give you the intended result:

grep -r myobject --exclude-dir=src/other/objects-folder

And you may wonder why exclude-dir isn't working! To actually exclude results from the objects-folder, simply do this:

grep -r myobject --exclude-dir=objects-folder

In other words, just use the folder name, not the path. Obvious once you know it.

From the man page:

--exclude-dir=GLOB
Skip any command-line directory with a name suffix that matches the pattern GLOB. When searching recursively, skip any subdirectory whose base name matches GLOB. Ignore any redundant trailing slashes in GLOB.

Answer 9

Very useful, especially for those dealing with Node.js where we want to avoid searching inside "node_modules":

find ./ -not -path "*/node_modules/*" -name "*.js" | xargs grep keyword

Answer 10

A simple working command:

root/dspace# grep -r --exclude-dir={log,assetstore} "creativecommons.org"

Above I grep for text "creativecommons.org" in current directory "dspace" and exclude dirs {log,assetstore}.

Done.

Answer 11

This one works for me:

grep <stuff> -R --exclude-dir=<your_dir>

Answer 12

find . ! -name "node_modules" -type d 

Answer 13

Step 1:

vim ~/.bash_profile

search() {
    grep -InH -r --exclude-dir=*build*  -e "$1" .
}

Step 2:

source ~/.bash_profile

Usage:

search "<string_to_be_searched>"

Answer 14

A simpler way would be to filter your results using "grep -v".

grep -i needle -R * | grep -v node_modules