How would you

@imdadcodes

52.4K followers · usually 1.8K views a post

Views

13.8K

Likes

324

Comments

36

Shares

7

Selling · SaaS › Developer Tool

—

Why it worked

The video addresses a common technical problem faced by developers (duplicate file detection) and offers a clear, step-by-step explanation of a more efficient solution using hashing functions. The use of visual aids and the promise of future content on related topics encourage engagement and follows.

Summary

The video explains how to check for duplicate files using code. It contrasts the naive byte-by-byte comparison with a more efficient method using hashing functions and digital fingerprints. For larger-scale applications like Dropbox, chunking is suggested as a superior alternative.

Structure

1Introduction to the problem of duplicate files
2Naive byte-by-byte comparison and its drawbacks
3Introduction to hashing functions and digital fingerprints
4Comparing fingerprints for duplicate detection
5Mention of chunking for larger-scale applications

Product placement

The video discusses file duplicate detection using hashing functions and mentions Dropbox as an example of a file upload service. It also shows diagrams illustrating file blocks, disk sectors, hashing algorithms, and chunking.

Call to action

make sure you give me a follow

More from @imdadcodes

Their other posts in the index, biggest breakout first.

On-screen text

DUPLICATE File Blocks 0 | 1 | 2 | 3 Disk Sectors File Blocks 0 | 1 | 2 | 3 Disk Sectors (mod 6) = 0 (mod 6) = 3 (mod 6) = 7 1055 Duplicates via Fingerprinting Idea: make duplicates fall into same bucket For each new document: compute its fingerprint large number, which is: - always same for two identical docs - hash collisions Complexity: O(nd) Original Message Block 0 Hash Block 1 Hash Block N Hash Hash Hash Final Hash

Transcript

How would you check that two files are the same as each other using code? This is a common problem if you're building a file upload service like Dropbox or even if you're building an application that needs to work with files and you want to avoid duplicates. The naive way would be to read both files byte by byte. The problem with that is it's pretty slow and it doesn't scale very well, especially if you're working with larger files. A better way is to use what's known as a hashing function. The idea is that this hashing function takes your file and produces a digital fingerprint. This fingerprint is like a signature, it's much smaller. And what you can do is you can compare both fingerprints to each other. If they're the same, you've got a match. And this works decently sized files. Now for your use case, a hashing function and fingerprint comparison will work pretty well. But if you're building something like Dropbox, you want to opt for what's known as chunking. I'll cover that in another video, so make sure you give me a follow.