Lnxgeek.org Wiki - Freedom of choice, I choose Freedom

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Intro ======
When dealing with the password hashes locally is faster to read them through a database. I have constructed a version where I use sqlite3 as the database. You can probably get better performance using a real database server.

I will use this sqlite3 database for other scripts found in this section.
===== Get the Hashes =====
To get hold of the hashes you can use one of the downloader versions out there, I have used this one: https://github.com/HaveIBeenPwned/PwnedPasswordsDownloader

Someone has conveniently made a Snap so on Ubunut you install it like this:
<code>
sudo snap install haveibeenpwned-downloader
</code>

Then you simply run:
<code>
haveibeenpwned-downloader 
</code>

to have it download a very big file with all the breached hashes in it.

===== Import Script =====

Here is a Python script which takes this big text file and import it into a sqlite3 database:

<file>
#!/usr/bin/env python3
import sqlite3
import sys

# Check if the text file's path is provided as an argument.
if len(sys.argv) < 2:
    print("Usage: init_db.py <path_to_textfile>")
    sys.exit(1)

# Constants for the SQLite database and the input text file.
DATABASE = 'leaked_hashes.db'
TEXTFILE = sys.argv[1]

def insert_hashes_from_file(filename, cursor):
    """
    This function inserts hashes from a given text file into the SQLite database.
    
    Parameters:
    - filename (str): The path to the text file containing the hashes.
    - cursor (sqlite3.Cursor): The SQLite cursor object for database operations.
    """
    print(f"Starting to insert hashes from the file: {filename}...")

    with open(filename, 'r') as f:
        counter = 0
        for line in f:
            hash_part, count_part = line.strip().split(':')
            cursor.execute('INSERT INTO hashes (hash, count) VALUES (?, ?)', (hash_part, int(count_part)))
            counter += 1
            # Print progress for every 1,000,000 records inserted.
            if counter % 1000000 == 0:
                formatted_counter = "{:,}".format(counter)
                print(f"Inserted {formatted_counter} records.")
                
    print(f"Finished inserting {counter} records in total.")
    
# Connect to the SQLite database.
conn = sqlite3.connect(DATABASE)
cursor = conn.cursor()

# Drop the hashes table if it already exists.
cursor.execute('''
DROP TABLE IF EXISTS hashes;
''')

# Create the hashes table in the database.
cursor.execute('''
CREATE TABLE hashes (
    hash TEXT PRIMARY KEY,
    count INTEGER
)
''')

# Insert the leaked hashes from the text file into the database.
insert_hashes_from_file(TEXTFILE, cursor)

# Commit the changes and close the database connection.
conn.commit()
conn.close()

</file>