> For the complete documentation index, see [llms.txt](https://d3ndr1t30x.gitbook.io/breaking-things-a-junior-pentesters-guide/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://d3ndr1t30x.gitbook.io/breaking-things-a-junior-pentesters-guide/sorting-files.md).

# Sorting files

A collection of commands I use to filter and sort files. Substitute the names of the files to suit your needs

```
cat fsocity.txt| sort -u | uniq > wordlist.txt
```

Here's a breakdown of the command

1. Reads the contents of the file "fsocity.txt" using the "cat" command.
2. Sorts the lines of the file in alphabetical order using the "sort" command.
3. Removes any duplicate lines using the "uniq" command.
4. Writes the sorted and de-duplicated list of words to a new file called "wordlist.txt" using the ">" redirection operator.

### Sorting files by line count in ascending order

```
find -type f -exec wc -l {} \; | sort
```

The given Bash one-liner performs the following tasks:

1. `find -type f`: This part of the command uses the `find` command to search for files (`-type f`) in the current directory and its subdirectories. It identifies all regular files in the directory tree.
2. `-exec wc -l {} \;`: For each file found by `find`, it uses the `-exec` option to execute the `wc -l` command, which counts the number of lines in the file (`-l`). The `{}` is a placeholder for the file name, and `\;` specifies the end of the `-exec` command.
3. `|`: This is a pipe symbol, which takes the output of the previous command (`find` with line counts) and passes it as input to the next command.
4. `sort`: Finally, the `sort` command is used to sort the list of file line counts in ascending order.

In summary, this one-liner searches for all regular files in the current directory and its subdirectories, counts the number of lines in each file, and then sorts the list of line counts in ascending order. The result is a list of file line counts in ascending order.

### Filtering Gobuster output

<pre class="language-bash"><code class="lang-bash"><strong>grep -E 'Status: (301|302)' gobuster.out | awk -F' ' '{print $NF}' > valid_urls.txt
</strong></code></pre>

Here's how this command works:

1. `grep -E 'Status: (301|302)' gobuster.out`: This uses `grep` with the `-E` option to enable extended regular expressions and looks for lines containing "Status: 301" or "Status: 302" in the input file `gobuster.out`.
2. `awk -F' ' '{print $NF}'`: This command uses `awk` to extract the last field (the URL) from each line, which is the part after the last space.
3. `> valid_urls.txt`: This redirects the extracted URLs to an output file called `valid_urls.txt`.

The `valid_urls.txt` file will contain a list of valid URLs that you can use for further analysis or testing.

### Printing and sorting files with awk

```
cat backup | awk '{print $7}' | sort
```

```
cat backup | awk '{print $7}' | sort | uniq -c| sort -n
```

```
find / -user shaun -ls 2>/dev/null|grep -v 'proc\|run\|sys'
```