CY 2550 - Foundations of Cybersecurity

Project: Password Cracking

This project is due at 11:59pm on Friday, February 26, 2021.

Description and Deliverables

In this project, you will gain hands on experience cracking passwords. We highly recommend that you start this project immediately: the necessary computations can take days to complete!

To receive full credit for this project, you will turn in the following:

  1. A file named project4/cracked.txt that contains the usernames and cracked passwords for the 60 users contained in the leaked /etc/shadow file that corresponds to your Northeastern username in the class Github.
This deliverable is described in greater detail below.

Password Cracking

Linux systems typically store cryptographically hashed user passwords in crypt format in the /etc/shadow file. If you have sudo access to a Linux system, you can view this file on your own system (don't try to look at this file on systems you don't own, like the Khoury College Linux machines). The file format for the /etc/shadow file is described here.

In this project, you will crack the hashed passwords contained in a "leaked" /etc/shadow file. Each student will get their own file (named your.northeastern.username.shadow) from a repository in the class Github. There are 60 usernames and passwords in the file, meaning that it will take several days of compute power to crack all 60 passwords, so start this process early! Each student will receive a unique /etc/shadow file, so there is nothing to be gained from collaborating with your classmates.

Cracking Tools

We recommend that students use well-known, heavily optimized cracking tools like John the Ripper or HashCat for this project. Both tools are available for multiple platforms, although they are trivial to install on Debian-based Linux systems: 
$ sudo apt install john
$ sudo apt install hashcat
On Macs, John and Hashcat can be easily installed using the Brew package manager: 
$ brew install john-jumbo
$ brew install hashcat
Both tools have built-in support for the /etc/shadow file format, have the ability to pause and resume cracking sessions (a useful feature, since cracking can take hours/days), and support multiple different strategies for guessing passwords (e.g., brute force, word lists, etc.). We leave it to you to determine which tool you prefer and learn its command line syntax. Students are welcome to use whatever password guessing approach they want; many wordlists are available for free online, including from the John the Ripper homepage.

John the Ripper and HashCat both have the ability to run in multi-threaded configurations (i.e., they try to crack multiple passwords in parallel). We highly recommend that students utilize these features; for example, on a quad-core laptop, running John the Ripper with the "--fork=3" option to use three CPU cores is a reasonable approach. Alternatively, if your computer has a GPU, we highly recommend using the GPU-optimized, OpenCL modes available in both programs, since GPUs are several orders of magnitude faster at password cracking than CPUs.

Cracking Approach

The leaked shadow file is designed to have a sliding difficulty scale. Without doing anything fancy, roughly one third of the passwords should crack in just a few minutes. Why do you think these passwords were so easy to crack?

With a reasonably comprehensive wordlist/dictionary (like those you used in this project) combined with common permutation rules, another third or so of the passwords should crack within 24 hours. For example, using John the Ripper the following command will attempt to crack the passwords using a wordlist of your choice and John's built-in permutation rules (e.g., capitalizing the first and last letters of words, adding random numbers to the end of words, etc.). 

$ john --wordlist=<path to your wordlist> --rules --fork=3 <path to the shadow file>
The remaining passwords are more challenging, and require more expansive permutation rules (hint: symbols) and raw brute force to crack. For example, using John the Ripper, you can attempt a brute force attack against the shadow file using all combinations of ASCII characters with length <14 using the following command: 
$ john --incremental=ASCII --fork=3 <path to the shadow file>
Note that this kind of brute force approach will take a long time to complete.

Just using the built-in permutation rules that come with John and Hashcat will not be sufficient to crack all the passwords; you will need to try additional, custom approaches. For example, Hashcat allows you to specify custom brute-force and hybrid attack modes on the command line.

Mask attacks allow you to customize how Hashcat attempts to brute-force hashes. It is different from the default brute forcing approach because with masks we seek to reduce the key space we use for a candidate password by specifying specific character-sets to try.

Let’s say you want to crack a hash and you know the original password is three characters long and consists of lowercase and uppercase letters. This password is not likely to appear in a dictionary so a mask attack is more appropriate. To do this, you need to fill three placeholders with a custom character-set. To create this custom character-set you use -1 and then specify that you want all uppercase letters (?u) and all lowercase letters (?l). The total key space for the custom character-set is now 52. You then need to specify how many placeholders to use this custom character-set for (three in this case). 

$ hashcat -m 500 <shadow-file> --username -O -a 3 -1?u?l '?1?1?1'
(Note the ... above requires all the basic arguments about where the shadow file is, etc.) Typical brute-force attacks would use all characters so by specifying a character with a smaller key space we reduce the total runtime.

Hybrid attacks allow you to combine both masks and wordlists. Let’s say you had a dictionary with just the word "northeastern". If the password you want to crack is "northeastern873" it would be very hard to do this with rules. However, in the hybrid attack mode a mask is simply appended to the word(s) from the given wordlist. In this case you would want to append three placeholders of the digits 0-9 (?d). 

hashcat -m 500 <shadow-file> --username -O -a 6 <dictionary-file> '?d?d?d'
If you don’t know where to begin, start with incrementing the length of your mask/hybrid attack in hashcat with -i to cover potential cases.

File Format

To complete this project, you will turn in a file named cracked.txt that contains the usernames and cracked passwords for the 60 users in the leaked shadow file. Each user and corresponding password should appear on one line in cracked.txt separated by a colon. For example, the format of a valid submission might look like this: 
cbw:really_strong_password6@
alice:1337cr4ck1ngsk1llz
bob:weak1234
charlie:lalala

Submitting Your Project

To submit your project, do the following:
  1. Create a directory ~/cy2550/project4 in the folder corresponding to your git repository.
  2. Copy your cracked.txt file to the ~/cy2550/project4 folder.
  3. Add these files to your repository, commit them, and push the committed files to Github.
  4. Submit your repository to Gradescope.

Grading

This project is worth 10% of your final grade, broken down as follows (out of 100): Points can be lost for turning in files in incorrect formats (e.g., not ASCII, missing usernames, etc.).

Tips