Page Scrapers

Hello, I’m excited to see an OSS alternative to Mint that I can self host. Definitely keeping my eye on this.

I’m aware that this project plans to offload a lot of automatic ingestion work to Plaid. But I think that kind of runs counter to the spirit of this project. i.e. You shouldn’t have to use a cloud service to access and manage your finances.

Right now, I’m fairly certain that Mint.com stores encryped passwords and fires up a web scraper to download transactions from most financial institutions.

If it’s good enough for Mint, why not for this project too? Since a lot of people will self host a local instance of Financial Freedom, storing encrypted passwords seems like it’s definitely an option.

I have professional experience building web scraping infrastructure if you’re into this idea and would like some help.

Hi @bradsk88 ,

It’s awesome that you are excited about Financial Freedom! We have a ton of stuff to add and can’t wait to see this grow!

I have the integration of Plaid as an immediate next step to transactions simply because of the vast support of financial institutions they provide and my lack of experience with web scraping development. However, I 100% agree with getting as much of your financial data from your institution → Financial Freedom.

Our overall goal was to first start with a database structure for Accounts + Transactions and continue to fine tune the relationships and calculations. I personally import from a CSV into Financial Freedom.

Next, I wanted to work on the queuing system that would grab data from Plaid automatically to keep transaction information up-to-date.

However, the goal would be to make Plaid just an option for people if that’s what they chose to use. I wanted to design that automatic interface in a way similar to a facade in Laravel, where users could bring their own provider (https://tink.com/, Plaid, etc.) and we can connect a variety of ways. This is where your web scraping could come in.

Honestly, I never knew how Mint got their data through web scraping, but it makes sense. If you would be willing to help, I’d love to hear your ideas!