For some time now I have been trying to solve the problem of organizing my computer files. I researched many options and techniques, and yet nothing really clicked with me. So, I finally decided to make my own system! The following is a description of the system I want to build. At the end of the article, I pose some yet unsolved questions and concerns. If you, the reader, would like to help with this project, reach out on HN! I would really appreciate it!
I generate a lot of files for various projects. Some of my projects are professional software and hardware development projects, or marketing and design projects, for ongoing or one-off customers. Others are personal video, welding, making, building, etc projects. Lot's of stuff to keep organized!
To maintain sanity I keep everything I generate, along with any long term reference materials and basically any files important to me, in a single root directory on my computer, currently in
Inside my Files directory I strive to maintain a tidy hierarchal structure. However, retrievability is still an issue. Allow me to illustrate with an example. Let's say I vaguely remember doing a project 2 years ago that used ESP32 (Micro Controller with BLE and WiFi) and an OLED screen. I want to find that project for reference, but I don't remember its name or the name of the client it was for. My professional projects are organized in a
/client/project hierarchy, which is of no help to me in this case. What I need is a way to search for projects that have the following criteria:
Sadly, in a purely hierarchical structure these relationships are impossible to represent without either links or tags. A project folder can only have one relationship with the directory hierarchy, it can have a parent folder.
Tags or links would help, but both are sporadically supported on major OSs, are not well supported by older filesystems, and are not at all supported by the cloud storage providers.
What we need is a cross platform, cross UI (GUI and CLI interface), and cross file system solution. It has to work on thumb drives with ExFAT filesystem. It has to preserve the metadata even when copied to Dropbox or Google Drive. And, it has to support Windows, Mac, and Linux.
After much research I ended up coming up dry. So it's time to make my own system. Here is what I came up with.
BinFinder is a desktop file organization tool. It organizes files into Bins and combines the following 3 major file organization paradigms into one system:
The base file organization unit in Bin Finder is a File Bin. A Bin is just a folder with a Bin Description File (file with
.bfb extension) inside the folder. The
.bfb file is a text file that stores all of the metadata about the Atom in JSON format. The metadata includes:
Key:Value schema, where
Key is the name of the tag and value is the value associated with that tag.
Because a File Bin is just a special type of folder, it can contain any number of files and sub-folders inside of it, including other Bins and Pointers to other Bins!
If you need to tag or point to an individual file, it can be wrapped individually in a Bin. Since most human endeavors of any substance produce multiple files, this use case is relatively uncommon.
Bins are organized into a Bin Tree. Basically a Bin Tree is just a root Bin with a bunch of other Bins, sub-folders, and files inside of it. Basically just a normal hierarchical filesystem.
You can add multiple Bin Trees to the BinFinder and search and filter across all of them. For instance, you can have your active project Bins live on your laptop and move them to the "Archive" Bin Tree that lives on an external USB Drive once you are done. Any pointers to the moved Bins will be updated with the new location of the Bins.
Pointers are just a fancy shortcut / alias / link to a Bin. What makes Pointers special is there ability to locate the Bin they are pointing too even if it has been moved or renamed.
Let's imagine you have a Project Bin in you "Active" Tree on your laptop. After finishing that project, to free up space, you move the project from the "Active" Tree to the "Archived" Tree. The "Archived" Tree lives on an external USB hard drive.
When you click on a Pointer to the Project Bin you just moved, the system will first check where it has seen the bin last. If the system can't find the Bin in that location, it will quickly rescan all of your Trees. If the Bin is found in another location, the system will update the pointer and open the Bin. If the Bin CANNOT be found, the system will inform you accordingly with a message similar to: "Can't Find Bin X! It was last seen inside 'Archived' Tree, which is not available currently." That's a helpful reminder to you that perhaps you need to plug in your USB SSD.
In practice a Pointer is nothing more than just a JSON encoded text file with
.bfp extension. It contains the following information:
BinFinder treats tags as basically additional metadata about the Bin. For instance, you can tag a bin with things like:
Client.Name : X Corp
Client.Contact : Dr. X from X Corp
Client.Email : email@example.com
Budget : 48,000
In the list above you'll notice that the first 3 tags are prepended with 'Client'. Those tags are automatically organized into the 'Client' tag bundle, and are grouped together in the UI for convenience.
The last tag,
Finished, has no value. To be more accurate, it has no value specified, so a default value of
True is auto assigned. Absence of a given tag is equivalent to having a
False value for that tag.
To bring it all together let's imagine a Bin for the project we used as an illustration in the 'Problem' section above. The bin lives at
~/Files/Clients/X Corp/Super BLE. Inside the bin path there is an
INFO.bfb file. That's the file that turns a simple old directory into a fancy Bin! Here is what it looks like:
To make it easier to find this project in the future I may also drop some Pointers to the Bin in various other places:
~/Files/Projects/By Year/2019/Super BLE.bfp
And here is what we would find in one of those pointer files:
When you double click on one of those
.bfp files, they are automatically opened with BinFinder. BinFinder looks for the Bin and opens it in your default file manager. In practice, the whole process should function just as if you clicked a shortcut.
I have most of the UI and workflows worked out in my head pretty well, but I find myself in a "paralysis by analysis" situation. Here are some of the questions I am struggling with in my head at the moment. Feel free to comment on HN and give me some ideas:
Finally, if you would like to help on this project, that would be really cool! Please feel free to reach out if you think it's a good idea, and let's build it together!