You might think so from only reading the title, but what the article/paper actually proposes is using CRDTs to create a synthesis of the advantages (for the user) of both cloud applications and traditional local applications.
Today's SaaS world is largely economically opposed to the idea of data ownership. It's a lot easier to make money by renting people access to their data.
The problem is not inherently technical. The solution must address the fact that the software businesses favor cloud solutions and other systems that make it difficult to stop spending money
Yes. I always think data freedom is more important than software freedom. For example it matters less that MS Word is not free as in freedom when you can open the file in something that is.
And cloud SaaS are enemies of both! After all, in order to have data freedom, you need to have something to open in that other program, and cloud solutions do their best to not give you proper open/save file (and even if there's an export function, and even if everything is actually included in the export, it often isn't followed by import function that could read that export).
For this reason, I avoid using cloud SaaS for anything where can avoid it.
Absolutely. There are some exceptions though. Github is the obvious one. Dropbox by nature has your data constantly exported. Google for its sins has Google takeout.
Yes. And the first two I use, treating them mostly as dumb pieces of infrastructure. Arguably, the functionality they provide have a crucial ops component that I'm all for paying for someone else to handle it for me. But neither Github nor Dropbox locks me into anything.
Google - yes, web e-mail obviously is similar to the above; as for their office suite, I recently found a good excuse to justify shelling out for a proper Microsoft Office subscription (though I don't like that it's a subscription), and I stick to using the faster, locally-available, file-using, much more powerful (if still proprietary) software.
Seperating the two so cleanly is inappropriate imho. Software is the access layer around data, hence
"Show me your flowchart and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious." -- Fred Brooks, The Mythical Man Month (1975)
I would even say that software freedom implies or even requires data freedom, while the reverse does not hold. As an example I don't think that Facebook or Google's data export tools make them exemplars of user choice or control.
We went from local, to local infrastructure/IT, to cloud services in which data is mixed with other data and ownership is nebulous.
The business as you say, favours this, because the 'downsides' of the model are more on the side of risk (your data leaking, losing rights to your data, your data being 'sold' etc.) - which we don't like to pay for until it's too late.
Unless there are 'big scares' or regulatory requirements etc. I don't see that much changing.
But a single major leak from Salesforce or say Google Docs could see a massive shift in how we think about such things.
any marginally successful "local-first" app is going to go and raise $10m in vc, switch to software as a service, and add an enterprise mode that requires user permissions and data access to be managed on the server
Sure, any one company probably will - but there’s a whole market.
As soon as that one company abandons the local-first model, a gap opens, which will (usually, eventually) be filled by a new company offering local-first until that new company does the same.
As long as the companies don’t band together and agree to end it, there should be a company offering that model somewhere somehow.
Even in the situation I described, the original company would leave the free / client side version on the site as free marketing. This is the standard today for enterprise-monetised open source software.
Solving this problem isn't about being local-first. It's about being local-last. You have to be able to make more money by selling a software license than you make by selling equity and chasing user acquisition and retention.
Then we'll see people waking up to the fact that all this proprietary data is a liability and subscriptions are golden handcuffs and people will finally get back to making real software again
I don't see what's wrong with that. Local First really just means distributed, fault tolerant, and eventually consistent but designed for user devices instead of a cloud "scale" service.
Why couldn't an enterprise run a "device" (a server) which others can easily sync to ("sync.enterprise.com") and which also only allows authorized users to access data which they're allowed to access? Maybe using Macaroons or something and devices can still sync locally via Bluetooth, Wifi, or whatever.
Now you have a full back up of everything on that server which IT could now more easily ensure it backed up, secured, and etc.
Not to mention the same idea could be used by a normal person just running a NAS at home or a server in DO/AWS/GCS/etc.
The blog post doesn't mention this, so I thought I'd point this out. One of the paper's authors is Martin Kleppmann, who wrote the very good https://dataintensive.net/ book.
Kleppmann co-created/major contributor to Apache Kafka along with Jay Kreps and Neha Narkhede. He also co-founded Rapportive, a YC company acquired by LinkedIn, along with Rahul Vohra, who is presently the CEO of Superhuman.
It’s strange that Evernote is omitted from the list - it is a great example of local-first app.
Their recent-ish history, when the their free tire become limited to only syncing a few devices, illustrates that even if software is fully local, and supports open formats, having the functional cloud matters, a lot.
I love Evernote and I use it daily, but Evernote lacks end-to-end encryption, which is a pity. I store lots of information in that app and I would be more reassured if I knew that it is only me who can read the data. I would even pay more in order to have that kind of encryption. I think that no current feature of Evernote would be affected by encryption as text recognition in images can be done at the client level.
I've been trying to document my "local-first" approach to managing photos. I've made it a ways through but am not sure when I'll finish. Posting here since it is relevant.
elodie looks amazing! I definitely need to try it out
it was just the thing I was thinking of building recently as it was getting really tiring to manually organize photos
one extra idea that I had: there's a cool project that would enable offline geocoding[1], which would help get rid of API limits while making the reverse geocode queries almost instant
(the included dataset is pretty limited, but it's not hard to extend from an openstreetmap planet dump)
I had looked into local geocoding databases but did not want to add a database file in the git repository. reverse_geocoder looks really interesting though and I may have a look at adding that.
For the time being, elodie does cache responses from the MapQuest API. And to reduce the number of API calls it does some approximation by seeing if an entry exists in the cache within 3 miles of the current photo --- if so, it uses that instead of looking up the location. [1]
This looks great and feels a lot like beets [1] for music, only that they use a database. I'll try it when I have the time to re-organize ~20 years of photos.
Just read through your google doc, interesting! But what about additional family members, with their own cameras, and no interest in any clever workflow activities :) I'm currently using Google Photos as my main service, and it's working good enough for now: each family member has the Google Photos app which uploads pics automatically, to their own account. We all share our Google Photos with each other. This way I (as main curator) have access to everyone's pics, without anyone having to do anything. Google lets you store the original size pics, so that is great (not like iCloud that resizes all pics!). Google also adds face recognition, which is very practical, and also provides a good interface for everyone to view the pictures. Regarding safekeeping: I use the Google Drive interface to backup all my photos to my local linux storage (combination of rsync and https://github.com/astrada/google-drive-ocamlfuse to mount Google Drive). This way I always have all original photos locally. Finally I backup everything offsite using BackBlaze.
All this relies heavily on Google Photos, but I have my own local backup of all original files. So if I need to change service, it should just be a one-time effort to migrate.
I'm using resilio sync on all phones to sync to a computer. All the family needs to do is start Sync and wait until it's done. Does not matter on what network they are on, it still syncs.
Uploading to both services is still supported through the Backup and Sync app. But once uploaded they are independent copies and deleting from one doesn't delete from the other. I also expect that this support will be deprecated at some point in the future.
Not keeping Drive and Photos in sync really killed it for me. I ended up switching from Google Drive to Dropbox but I still use Google Photos.
I have photos added to my library in Dropbox automatically added to my Google Photos library and this has worked well so far. [1]
I also rely mostly on Google Photos as my viewing and sharing app. This includes sharing photos with family which is done through Google Photos shared albums.
I'm really the only one who cares about archiving photos so I'll transfer the shared photos from Google Photos to Google Drive (using the "share" functionality from the mobile app).
This kicks off a workflow that simultaneioously organizes the shared photo into my library and copies it too Dropbox and uploads it to my Google Photos library [1]. (I use Google Drive as a transport mechanism to get photos off my phone and onto my Synology).
Not ideal but once I got it set up it's worked really well.
Things are happening in that space already!
I work at Actyx and we have a production ready stack for local-first real serverless (peer-to-peer) applications. Please take a look at https://actyx.com/ .
P2P has miles to go before challenging the reliability, convenience and performance of the cloud.
That said, one area undervalued is Partially homomorphic cryptosystems[1], where the cloud never ever gets to see unencrypted user data.
I hope the future is fast-local compute on cached data, with the cloud holding a much larger, encrypted but permissioned data store, offering utility functions like search over encrypted data
Martin Kleppmann is a major inspiration for our startup, Ditto.
We take the local-first concept and p2p to the next level with CRDTs and replication. But what we really do is leverage things like AWDL, mDNS, and or Bluetooth low energy to sync mobile databases instances with each other even without internet connectivity. www.ditto.live
I made "traditional" enterprise-style-apps for small business and have tried to crack the sync of data several times.
Exist a resource in how leverage that tech with boring stuff like inventory, invoices, etc? Hopefully,. without a total change of stacks (I use postgresql, sqlite as dbs, and need to integrate with near 12+ different db engines)
The data ownership is very important for business world. The reason why we built our products for IoT ( www.bevywise.com ) as a more install able version is the data privacy and ownership.
We really see this largely in manufacturing industry. If not local, we should provide private servers and data security.
Syncthing solves a large part of syncing data between devices using your own VPS, server(s), etc. If your VPS provider goes out of business, you can then just fire up a new VPS and hook it back up to your local machine(s).
I'm on Android so it works, but I also used Nextcloud for some time and had issues with it "forgetting" about the data folder, so I had to re-create it every time it happened. I wasn't happy about that.
there's also resilio sync that gives you the option have the node/folder on your vps be encrypted. hopefully syncthing will add that feature at some point in the future
I have been considering spending more money than I should finding someone on a freelancer site to add the untrusted node / encrypted file transfer but don't give a certain node the key to decrypt the file support, that is the number one missing feature for syncthing for me.
It would unlock scenarios like share your files with me and I'll share files with you and neither of us will know the others files unless your house burns down in which case you can get your files back just by hookin the sync back up with the encryption key pulled from your password manager.
Cryptomator [1]. Cross-platform, allows you to encrypt your data in the cloud, and access it transparently.
Thing is, like Syncthing, it lacks a collaborative feature. Nextcloud has it, but only if you have the Nextcloud accessible (I want to host only on LAN). Something like IPFS (or Tor) is a solution to such problem.
When I select software these are among the list of things I am looking for generally:
- file formats that won’t lock you in or are even openly hackable (allows you to automate things)
- no clouds that will break the software once it is gone
- local storage with custom syncing or backup options
- strictly no weird data collection or “We own the rights to your data”-Type of terms
So if I get the slightest feeling of a lock in or unnecessary data collection you are scaring me away, because mentally I would then already look at the time after you decide to scrap your cloud or abandon your file formats. The data collection bit shows me your users aren’t front and center but something else is which makes your product even less of a good choice.
If your product runs on the web, allowing for self-hosted solutions is also a big plus.
While I fully agree with your selection criteria, please consider the other side of the equation, because engineering (and the world) is all about compromises.
I am the author of a SaaS app (https://partsbox.io/). I export in open formats (JSON), there is no lock-in, it's easy to get all of your data at any time. But the app is online and will remain so. Why? Economics. Maintaining a self-hosted solution is an enormous cost, which most people forget about. You need to create, document, maintain and support an entirely different version of your software (single-user, no billing/invoicing, different email sending, different rights system, etc). And then every time you change something in your software you have to worry about migrations, not just in your database, but in all your clients databases.
I am not saying it's impossible, it's just expensive, especially for companies which are built to be sustainable in the first place (e.g. not VC-funded). Believe me, if you don't have VC money to burn, you will not be experimenting with CRDTs and synchronizing distributed data from a multitude of versions of your application.
I regularly have to explain why there is no on-premises version of my app. The best part is that many people think that an on-premises version should be less expensive than the online version, and come without a subscription.
> The best part is that many people think that an on-premises version should be less expensive than the online version, and come without a subscription.
Which makes sense IMHO, provided they are not expecting any updates to their on-prem installation. It can just be a fork of your current codebase with no new features or warranty. Maybe you can include some terms for critical security updates but that is about it.
Agreed. But if a user agrees to be left "out in the cold", you should allow him to. Given that it is not very costly for you to prepare your software for on-prem, it maybe a win-win for all the parties involved.
Assuming you are not providing any updates (not even security critical ones for argument's sake), you only need to package and add documentation for an on-prem installation. Why is it so costly?
FUD. Among my products I have desktop software. Sure I got some pains upon initial release but it's been literally years since I really needed to test "for million configurations".
I also have experience developing and maintaining cloud solutions. Total amount of work that goes into large cloud apps and amount of things things that can brake or not work properly is fairly impressive. Definitely not any less then on premises
I think we as engineers owe our users bug fixes, because we messed up at some point.
However new functionality or email/phone support? Well, this is what people are not entitled for.
Maybe you should look in the mirror. If you are actively developing software and adding new features - sure, you might be also developing critical bugs.
In my case for example I have few older products (desktop apps that still sell and bring some money) for which I have source code that has not been touched for years. Not interested in development new features either as those apps are mature and there is not much ROI in developing new features the whole 2.5 customers are willing to pay for.
Long story short I do not have any critical bugs there. Maybe do exist and hiding somewhere but since nobody had ever discovered them I am totally cool. The only occasional customers complaint I have every once in a while is RTFM type. Not real bugs.
That's not an unreasonable assumption, right? It's just an extrapolation from how desktop/local software has always worked. Demanding a subscription for something that doesn't, technically, need any maintenance or ongoing costs on your part is a relatively new thing. Yes, of course, people expect a stream of updates these days for security if nothing else. But you could sell a version of your app that's static - urgent bug fixes only - and then sell it to them again when a new major version is released, old school style.
A lot of devs have moved away from that because subscriptions provide better peace of mind and smooth out the income stream, but it's not clear it's really better for users. Certainly they lose some optionality and there's less market pressure on the suppliers to ship big new features that motivate upgrades.
I believe a one-time fee for software is fundamentally unsustainable, and a relic of the past, when we had no networking and our operating systems did not evolve quickly. Every piece of software needs maintenance and support.
Also, a lot of people seem to forget that even if we pretended that support isn't necessary, just the existence of a standalone version imposes ongoing costs: there is more testing, and the scope of changes that can be made is more limited.
Subscriptions are the only sustainable way of maintaining software in the long term. We can either accept that and move on, or keep pretending we "buy" software that will work forever, and then pay every year for a "major new version", which apart from the fundamentally fictional nature of the deal, results in developers cramming in new and unnecessary features instead of focusing on software quality.
I could be a potential customer, since my company works on the embedded market and we design and manufacture all our PCBs.
There was a time the company has a moment of big growth, and we looked into solutions like yours. The question was: what would happen if this guy (not specifically you, others similar to you) closes tomorrow? The answer generally was: you can export all your data and then you feed it into your local database. So the matter ended with: OK, we'll stick with our current database. Goodbye.
At the end of the day we saw such services as a glorified data-entry/store/search database that hardly will suffer many modifications or require updates, and since pretty much of our distributors (apart from mouser, digikey, avnet, etc.) won't support data exchange and PO/Offers will be negotiated in the traditional way (also negotiating price reductions). No need for VC funding or anything else.
We need to be able to open an schematic/BOM/whatever in 10 years and be able to operate as if it was just created yesterday, so much of the software we use is "a relic of the past" (in SaaS vendors eyes), but effective and "the way is supposed to be".
Online/cloud or subscription-based CADs? Also kicked out of the door.
With this I want to say that people like the article author seem to live in a little box, thinking on what they use daily as web-service-saas-whatever developers, without realizing the world is much bigger than they think it is.
"...I believe a one-time fee for software is fundamentally unsustainable..."
It is pretty much sustainable. Except few natural cases SAAS is just a wet dream of vendors trying to have constant revenue stream the easy way.
If software needs support/new features you pay for upgrades. Or not if the old version works just fine for you. In my case I use tons of software for which I only paid once (sometimes I pay for upgrades as well). If I had to pay monthly fee for each title I use I'd be spending insane amount of money.
> I believe a one-time fee for software is fundamentally unsustainable
I strongly disagree. I have always, up to this day, sold software on that model, and I will probably always do so. There's nothing unsustainable about it.
What is true is that rent-seeking can be much more profitable. But fortunately for those of us who find it very distasteful, it's not necessary.
one-time fee only works if you have a constant stream of new customers into your niche that is sufficient to pay your salary.
with upgrades or subscriptions you can have a group of users who love what you do and are happy to support you without having to hope for new users and constantly market to get them.
> what do you do when you've saturated the market?
If I want to continue active development on it, then I sell upgrades. If not, then I'll continue to do free maintenance releases, but my main business will be from different products.
> one-time fee only works if you have a constant stream of new customers into your niche that is sufficient to pay your salary.
You can continue to get income from existing customers. You just have to provide real value to them in exchange for it.
Worth a thought. How does desktop software achieve backwards compatibility? For example Libreoffice can work with arbitrary datastores from the 1990's. Meanwhile with modern web based software we struggle to maintain compatibility within a single datastore.
You raise a valid point. I think what it ultimately boils down to is to give your users the feeling that even if your service stops (and there are many reasons why it could), all the hours they put into your product are not lost. The criteria I listed are factors after all, and how I factor them in depends on the available alternatives and the work I will put into it myself and whether a online-solution actually makes sense there.
A good example are note taking apps. My notes should be private and I want to be able to read them ten years later. For me your product would need to add something valuable to this that a filesystem and a bunch of files that I synchronize myself can't do. As for now there is no note-taking app I found where the benifits outweight the preceived loss of privacy and reliability. The online thing can make sense, but syncing my phone with Nextcloud works even better, so I don't really see why I'd need it.
This is potentially different with an app like yours, because the benefits of using your app vs using e.g. a spreadsheet seem truly tangible. Using JSON and allowing export at any time is a huge plus. Having it a web-only kinda makes sense, as your app seems to be geared toward teams (and any serious parts managment makes a lot more sense when you are not alone).
While the additional work that would have to go into documentation and programming if you were to offer this as a self-hosted variant is non-trivial, from my standpoint offering the option to self-host can also be seen as an act of communicating: "Don't worry, whatever happens to this project, you will not lose the time invested".
I am not obsessed with this kind of reliability, but I just want to avoid the future hassle of having to deal with this, especially when I put a lot of time into it.
> Believe me, if you don't have VC money to burn, you will not be experimenting with CRDTs and synchronizing distributed data from a multitude of versions of your application.
While you're right that doing this is less convenient, I can say with the personal experience of developing a few such projects on a shoestring that doing this isn't as hard or expensive as you're making out. It's entirely doable. It just requires thoughtful engineering.
A lot of software doesn’t have to be “hosted” anywhere. The first question I usually have when trying out some web-based software is: “Why on earth can’t this be a native application that I just download and run? Why are they shoving a web browser and server into this thing?” With a cross-platform native application, you bypass the whole on-prem vs hosted question entirely.
I wonder how often the answer is simply “We know lots of JavaScript programmers and not a lot of Qt programmers.” Talk about the tail wagging the dog.
As a user, I strongly favor software I buy once, download, and run as-is, forever, untethered to the Internet.
i agree but would add that there are a number of reasons why your browser as the client is a better long term strategy that minimizes maintenance. Unless you want a really generic geeky bare-bones UI. Native UI toolkits don't have a long lifespan. You constantly need to tweak what you've built for changes in the latest version of the OS.
However, that doesn't mean it needs to be "hosted" anywhere. It's trivial to make an app run a little web server that they access with their browser instead of a native or QT UI.
Browser based interfaces are an excellent bet (probably the best bet) for "run as-is, forever" or as close to "forever" as we can reasonably get right now.
> Maintaining a self-hosted solution is an enormous cost, which most people forget about. You need to create, document, maintain and support an entirely different version of your software (single-user, no billing/invoicing, different email sending, different rights system, etc).
I think you're not looking at this from a local-first perspective. From that perspective you can have the same app locally as on the server. There's only one version. Yes, it does require more planning and atypical approaches, but it's 100% doable.
I'm in the planning stage of a local first web-app that will have a server side version, and it's literally going to be the exact same code on both.
I can see some arguments for having _slight_ differences between server and client software but nothing that isn't easy for a solo-dev to maintain. Mostly set it up once and never touch it again type things.
thinking local-first has fundamentally changed some decisions i normally make without thinking for web apps, but I think it will absolutely be worth it for my users in the end.
i used resilio for years and thought the same about syncthing until i switched last year and now I actually prefer how syncthing is setup.
if you pay for resilio there is an option to add all your folders in one go but on some computers I don't want to add all of them anyway so that's not much use to me.
with the free version you have to manually add folders one by one but to do that you need the key which means you need to copy them to a text file and add them on another computer.
with syncthing, it will detect other syncthing devices on your home network so you just have to add the ones you want then accept the request from another device.
once that is done you select which folders the device has access to and then a notification will show up on said device asking you to connect. so basically no fiddling with keys or having to store them somewhere secure
(this is all presuming i was using resilio correctly, maybe there was an easier way I was not aware of)
Interesting to see this. We've taken a similar view for the initial phase of the micro network. Locality is going to matter more and more as we move into the future. Although Cloud still has its place and we don't ignore that either.
You don't have to invent entire new paradigms such as CRDT for this. Unix is all about site autonomy, no-BS tooling, simplicity, and portability. So for your next project, consider Unix/Linux as deployment target during development, and only then deploy it to a cloud-hosted Unix cluster, with a local-first but cloud-hosted DB such as PostgreSQL and standardized middleware such as AMQP/RabbitMQ/qpid rather than provider-specific solutions, or at least use de-facto standard protocols such as s3 and MongoDB (if needed) and supported by multiple clouds. Many people are prematurely committing to k8s and "microservices" but in my experience, even though k8s as such isn't intended as a lock-in strategy, it has the effect of absorbing so much energy in projects (with devs more than happy to spend their time setting up auth, load balancing, and automating things rather than on business functionality), and then still ending up with a non-portable, incomprehensible mess of configs and deploy scripts that it just isn't worth it.
My view on this is a bit different. I see Kubernetes as the abstraction layer on top of the cloud providers. In the last few years I have set up multiple k8s clusters for clients who specifically do not want to be locked in to a certain cloud provider. Once the software is running on top of k8s it is easy to switch cloud providers without changing the software.
Switching to another cloud provider this way is trivial and usually only involves changing the Terraform configuration to setup a k8s cluster on another cloud. All k8s-specific config/deploy files can be reused on the new cluster.
This of course only works if (as you suggest) you stay away from cloud-specific services (SQS, aurora, ECS, S3) and run everything in-cluster, or use managed services that are available on multiple providers (Postgresql via RDS, or Digital Ocean managed Postgres, Cloud SQL on GCP)
> Switching to another cloud provider this way is trivial
Based on my limited experience, I highly doubt this. Have you actually deployed cross-cloud k8s setups, or is this merely a theoretical statement on your part? Deploying to another cloud provider brings a whole new universe of failure modes and auth quirks, let alone migration and switch-over woos.
I’ve done a couple cluster switches from k8s on AWS to other providers like Digital Ocean and GCP. As far as I recall we had no issues and one of those was done in about an hour were most of the time was spent waiting for pg_dump/restore.
Note that most of these were not production clusters so switch-over was just data restore and DNS changes.
I build clusters from the start to not use cloud-specifics where possible and all cloud-specific configuration is on the cluster edges in terraform which you have to rewrite anyway when switching clouds.
Auth things like IAM permissions are not an issue if everything is “in cluster” and auth/permissions are checked there.
Most of these deployments consist of several application servers, PG databases, redis, rabbitmq etc
>It should be fast. We don’t want to make round-trips to a server to interact with the application.
The cloud apps are not slow only because of moving data, but there is also a problem that an average server is fast(16cores CPU + 64GB RAM), but If it's used by let's say 100users, It means one user has only 0.16core + 0.64GB memory. So an average laptop(4cores/4GB) or phone(4cores/1GB) is way faster. Basically people buy billions of transistors to use them only as a terminal to the cloud. Not to mention privacy risks.
A week ago, I did showHN for skyalt.com. It's a local accessible database(+ analytics, which is coming soon). I'm still blown away how fast it is, that you can put tens of millions of rows to single table with many columns on consumer hardware and you don't pay for scale or attachments.
That's not a fair comparison since most of the memory usage is just loading the app in to memory and then everyone is sharing the same app already loaded. Web apps don't have to be nearly as slow as they are. It's just that it's easier to make a slow app than a fast one. Also desktop apps are becoming super slow and bloated now thanks to electron.
Sad about the Electron route. I don't consider it to be "desktop" software. Native toolkits are miles faster than the extremely lazy approach of running an entire new browser instance because someone wanted to write javascript for their "app". It's terrible.
You have to consider the costs of building an app native for each platform vs a cross platform framework built with javascript. Single javascript code base for all platforms is a lot cheaper.
I am not sure on that really. I built a cross-platform control software for high-end audio equipment that ran on macOS and Windows and did all of its own drawing (charts, graphs, ALL controls [knobs, buttons, grids, faders etc etc etc], 2D overview with links of all devices, 3D scene of venue layout with OpenGL, diagrams, alerts and floating popup windows). No bitmaps used anywhere, all generated in code. Another guy handled the network stack that it used which was an entirely-bespoke protocol and he wrote the control firmware on the equipment.
This was in C++ with wxWidgets. It redrew every 50ms and logged data received so you could keep historic data of all devices on the network.
I am pretty sure that all of these giant companies releasing garbage Electron apps have a bigger budget than the budget on the 2 people (us!) that wrote what we wrote in 2.5 years.
When I see how abysmal apps like Skype and Slack are, I despair. Colossal amounts of RAM and simply displaying text and pictures.
It might be more convenient for the developer to write in the first language they learned but it is producing giant bloated applications. More CPU cycles, more allocations, more power consumed, shorter battery life on mobile devices (laptops), more charging of devices, more fossil fuels burned.
> Also desktop apps are becoming super slow and bloated now thanks to electron.
I can't quite get this point. From my perspective software engineers love/adore electron applications.
Look at VScode as the example:
- electron based javascript application
- telemetry included
- proprietary build with "open core"
It is literally the most popular code editor right now (p.s. I don't use it). Why as a tech savvy user you will use something you don't like for 5-10 hours each day to do your work?
Only answer I can see that electron is not an issue here.
Obvious answer: it's the only IDE-like thing that's both well supported and caters to the huge population of JavaScript-only developers, who generally don't want to use IntelliJ or similar products because they can't customise it using only web-stack skills.
That is, if JetBrains had made JS plugins first class citizens of their products, possibly VS Code wouldn't be as popular as it is.
VSCode is proof that an Electron app can be performant, but that's not the rule, and most people writing Electron apps aren't as talented as the VSCode team.
Also, there's an expectation that an IDE will be somewhat heavyweight. I don't mind if VSCode or IntelliJ grabs a few gigs or RAM, because I live inside those applications and depend on the features they provide.
What I don't want or need is Yet Another Electron Based Markdown Editor that gobbles up half my laptop's memory so that I can edit a hundred line text document.
As a user of VSCode, I use it in spite of it being based on Electron because there are no other good alternatives. It is noticeably very slow on many tasks, and has a large startup time. I recently had it basically become unusable when opening a 2MB yacc file and had to switch to Sublime Text to edit the file. I much prefer Sublime Text in terms of speed, but VSCode simply has more and better extensions for different programming languages making it better for day-to-day coding. If somebody were to make a performant text editor that has similar features to VSCode I would switch in a heartbeat.
I was a strong supporter of VSCode for a long period of time. Now I realise there is always an alternative.
After taking time to learn my tools I can't use vscode anymore because of how inefficient and restrictive it is now for my workflow. I mainly prefer tools that can last a lifetime and time you've investend now can yeld much better results across several decades of usage.
Here is short list of tools I can't live without: magit, org-mode, undo-tree (actualy it was a feature request and vscode team said it's too compticated for broad audience), ability to hack your code editor as you wish. Ability to work all day without touching a mouse once.
I do not love electron applications. They gobble RAM. Allocation is the enemy of performance.
I stopped using VS Code because of the billions of files it ships with (does anyone remember DLLs these days or what?) and its performance and debugging capabilities were very disappointing.
> If it's used by let's say 100users, It means one user has only 0.16core + 0.64GB memory. So an average laptop(4cores/4GB) or phone(4cores/1GB) is way faster.
This is overly simplistic. You're pretending that cores/memory are "allocated" to users, but really, a user might only make a few tens of requests, and the server only needs to spend a second or two servicing each request. On a server with only 100 users, it could very well be the case that a user has all 16 cores + 64gb available at the time they make a request. Also, as another commenter pointed out, you could use a large chunk of that memory for shared resources, and then each request might only need a few mb of memory to service a request.
I love this idea. Especially the end-to-end encryption for data that passes through a server to enable the ease of cloud computing without relinquishing data ownership.
It also depends on _who_ owns the data. In an enterprise environment the company usually has a vital interest in the data and on-premise deployments are a good way of retaining cloud computing without giving up data ownership. I'm surprised that more SAAS products don't offer on-premise given the privacy and ownership benefits. The tricky part there is making software that is easy to deploy and maintain, which might be the reason that it isn't done more often.
A product like Grammarly that allowed on-premise deployment would side-step a lot of the issues with sending all that data to a third party. I can't imagine a law firm ever being able to (legally) sign up for that.
Maybe on-premise installation happen but are not advertised? I have experienced one case in a previous company of a deal large enough to justify one for a client with sensitive data. No need to say operations were not happy, but I don’t have much informations to judge if it was a good deal for the company in the end.
Those 7 points remind me of the Syncdocs https://syncdocs.com app which ticked most of them. Using an encryption app and Dropbox will let you do the same thing, too.
I learned of Plan-Systems.org, they’re working towards something like this. Their company is non-profit, their collaboration tools are open source and protocol and the service is built on Unity and Unreal, which makes it cross-platform.
245 comments
[ 5.3 ms ] story [ 270 ms ] threadI really love making apps this way for some reason. I think it’s the focus on just the UI and not worrying about the back end until later.
For this particular app I’d consider “smartwatch first” to have been better as its for fitness!
The problem is not inherently technical. The solution must address the fact that the software businesses favor cloud solutions and other systems that make it difficult to stop spending money
For this reason, I avoid using cloud SaaS for anything where can avoid it.
Google - yes, web e-mail obviously is similar to the above; as for their office suite, I recently found a good excuse to justify shelling out for a proper Microsoft Office subscription (though I don't like that it's a subscription), and I stick to using the faster, locally-available, file-using, much more powerful (if still proprietary) software.
We went from local, to local infrastructure/IT, to cloud services in which data is mixed with other data and ownership is nebulous.
The business as you say, favours this, because the 'downsides' of the model are more on the side of risk (your data leaking, losing rights to your data, your data being 'sold' etc.) - which we don't like to pay for until it's too late.
Unless there are 'big scares' or regulatory requirements etc. I don't see that much changing.
But a single major leak from Salesforce or say Google Docs could see a massive shift in how we think about such things.
As soon as that one company abandons the local-first model, a gap opens, which will (usually, eventually) be filled by a new company offering local-first until that new company does the same.
As long as the companies don’t band together and agree to end it, there should be a company offering that model somewhere somehow.
Solving this problem isn't about being local-first. It's about being local-last. You have to be able to make more money by selling a software license than you make by selling equity and chasing user acquisition and retention.
Then we'll see people waking up to the fact that all this proprietary data is a liability and subscriptions are golden handcuffs and people will finally get back to making real software again
I've been doing some work on it over the last few days: https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-b...
Why couldn't an enterprise run a "device" (a server) which others can easily sync to ("sync.enterprise.com") and which also only allows authorized users to access data which they're allowed to access? Maybe using Macaroons or something and devices can still sync locally via Bluetooth, Wifi, or whatever.
Now you have a full back up of everything on that server which IT could now more easily ensure it backed up, secured, and etc.
Not to mention the same idea could be used by a normal person just running a NAS at home or a server in DO/AWS/GCS/etc.
<3 https://www.youtube.com/watch?v=fU9hR3kiOK0
Their recent-ish history, when the their free tire become limited to only syncing a few devices, illustrates that even if software is fully local, and supports open formats, having the functional cloud matters, a lot.
But yeah otherwise Evernote is pretty good for a single user.
https://www.macworld.com/article/1058169/mcnucle.html
A Pragmatic Photo Archiving Solution: https://docs.google.com/document/d/1JzqT-DJFlS2e8ZC00HrsQITq...
It's the culmination of software I've written [1] + a workflow that's resulted from it [2, 3, 4, 5].
[1] Elodie - https://github.com/jmathai/elodie
[2] Understanding my need for an automated photo workflow - https://medium.com/vantage/understanding-my-need-for-an-auto...
[3] Introducing Elodie; Your Personal EXIF-based Photo and Video Assistant - https://medium.com/vantage/understanding-my-need-for-an-auto...
[4] My Automated Photo Workflow using Google Photos and Elodie - https://medium.com/@jmathai/my-automated-photo-workflow-usin...
[5] One Year of Using an Automated Photo Organization and Archiving Workflow - https://artplusmarketing.com/one-year-of-using-an-automated-...
it was just the thing I was thinking of building recently as it was getting really tiring to manually organize photos
one extra idea that I had: there's a cool project that would enable offline geocoding[1], which would help get rid of API limits while making the reverse geocode queries almost instant
(the included dataset is pretty limited, but it's not hard to extend from an openstreetmap planet dump)
[1] https://pypi.org/project/reverse_geocoder/
For the time being, elodie does cache responses from the MapQuest API. And to reduce the number of API calls it does some approximation by seeing if an entry exists in the cache within 3 miles of the current photo --- if so, it uses that instead of looking up the location. [1]
[1] https://github.com/jmathai/elodie/blob/75e65901a94e14e6fd1ff...
[1] http://beets.io/
All this relies heavily on Google Photos, but I have my own local backup of all original files. So if I need to change service, it should just be a one-time effort to migrate.
I thought this was killed recently[1]
(this = google photos appearing in google drive)
Does it still work for you?
If yes, how? :)
[1] https://www.blog.google/products/photos/simplifying-google-p...
Not keeping Drive and Photos in sync really killed it for me. I ended up switching from Google Drive to Dropbox but I still use Google Photos.
I have photos added to my library in Dropbox automatically added to my Google Photos library and this has worked well so far. [1]
[1] https://github.com/jmathai/elodie/tree/75e65901a94e14e6fd1ff...
I'm really the only one who cares about archiving photos so I'll transfer the shared photos from Google Photos to Google Drive (using the "share" functionality from the mobile app).
This kicks off a workflow that simultaneioously organizes the shared photo into my library and copies it too Dropbox and uploads it to my Google Photos library [1]. (I use Google Drive as a transport mechanism to get photos off my phone and onto my Synology).
Not ideal but once I got it set up it's worked really well.
[1] https://github.com/jmathai/elodie/tree/master/elodie/plugins...
That said, one area undervalued is Partially homomorphic cryptosystems[1], where the cloud never ever gets to see unencrypted user data.
I hope the future is fast-local compute on cached data, with the cloud holding a much larger, encrypted but permissioned data store, offering utility functions like search over encrypted data
[1]: https://en.m.wikipedia.org/wiki/Homomorphic_encryption
We take the local-first concept and p2p to the next level with CRDTs and replication. But what we really do is leverage things like AWDL, mDNS, and or Bluetooth low energy to sync mobile databases instances with each other even without internet connectivity. www.ditto.live
Check it out in action!
https://youtu.be/1P2bKEJjdec https://youtu.be/ITUrk_rjnvo
We found that CRDTs, local first, and eventual consistency REALLY shines in the mobile phones since they constantly experience network partitions.
Exist a resource in how leverage that tech with boring stuff like inventory, invoices, etc? Hopefully,. without a total change of stacks (I use postgresql, sqlite as dbs, and need to integrate with near 12+ different db engines)
We really see this largely in manufacturing industry. If not local, we should provide private servers and data security.
Syncthing solves a large part of syncing data between devices using your own VPS, server(s), etc. If your VPS provider goes out of business, you can then just fire up a new VPS and hook it back up to your local machine(s).
With Syncthing I don't (yet) have this problem.
It would unlock scenarios like share your files with me and I'll share files with you and neither of us will know the others files unless your house burns down in which case you can get your files back just by hookin the sync back up with the encryption key pulled from your password manager.
Thing is, like Syncthing, it lacks a collaborative feature. Nextcloud has it, but only if you have the Nextcloud accessible (I want to host only on LAN). Something like IPFS (or Tor) is a solution to such problem.
[1] https://cryptomator.org/
It does not, you can share folders with anyone you want without them even needing an account.
- file formats that won’t lock you in or are even openly hackable (allows you to automate things)
- no clouds that will break the software once it is gone
- local storage with custom syncing or backup options
- strictly no weird data collection or “We own the rights to your data”-Type of terms
So if I get the slightest feeling of a lock in or unnecessary data collection you are scaring me away, because mentally I would then already look at the time after you decide to scrap your cloud or abandon your file formats. The data collection bit shows me your users aren’t front and center but something else is which makes your product even less of a good choice.
If your product runs on the web, allowing for self-hosted solutions is also a big plus.
I am the author of a SaaS app (https://partsbox.io/). I export in open formats (JSON), there is no lock-in, it's easy to get all of your data at any time. But the app is online and will remain so. Why? Economics. Maintaining a self-hosted solution is an enormous cost, which most people forget about. You need to create, document, maintain and support an entirely different version of your software (single-user, no billing/invoicing, different email sending, different rights system, etc). And then every time you change something in your software you have to worry about migrations, not just in your database, but in all your clients databases.
I am not saying it's impossible, it's just expensive, especially for companies which are built to be sustainable in the first place (e.g. not VC-funded). Believe me, if you don't have VC money to burn, you will not be experimenting with CRDTs and synchronizing distributed data from a multitude of versions of your application.
I regularly have to explain why there is no on-premises version of my app. The best part is that many people think that an on-premises version should be less expensive than the online version, and come without a subscription.
Which makes sense IMHO, provided they are not expecting any updates to their on-prem installation. It can just be a fork of your current codebase with no new features or warranty. Maybe you can include some terms for critical security updates but that is about it.
I believe that the whole concept of "software without support" is fundamentally flawed.
I have been asserting exactly the opposite in this thread.
None of which are insignificant.
In addition to that you must now support (and test for) a million configurations, rather than just one.
I also have experience developing and maintaining cloud solutions. Total amount of work that goes into large cloud apps and amount of things things that can brake or not work properly is fairly impressive. Definitely not any less then on premises
But they will expect it.
In my case for example I have few older products (desktop apps that still sell and bring some money) for which I have source code that has not been touched for years. Not interested in development new features either as those apps are mature and there is not much ROI in developing new features the whole 2.5 customers are willing to pay for.
Long story short I do not have any critical bugs there. Maybe do exist and hiding somewhere but since nobody had ever discovered them I am totally cool. The only occasional customers complaint I have every once in a while is RTFM type. Not real bugs.
A lot of devs have moved away from that because subscriptions provide better peace of mind and smooth out the income stream, but it's not clear it's really better for users. Certainly they lose some optionality and there's less market pressure on the suppliers to ship big new features that motivate upgrades.
Also, a lot of people seem to forget that even if we pretended that support isn't necessary, just the existence of a standalone version imposes ongoing costs: there is more testing, and the scope of changes that can be made is more limited.
Subscriptions are the only sustainable way of maintaining software in the long term. We can either accept that and move on, or keep pretending we "buy" software that will work forever, and then pay every year for a "major new version", which apart from the fundamentally fictional nature of the deal, results in developers cramming in new and unnecessary features instead of focusing on software quality.
I could be a potential customer, since my company works on the embedded market and we design and manufacture all our PCBs.
There was a time the company has a moment of big growth, and we looked into solutions like yours. The question was: what would happen if this guy (not specifically you, others similar to you) closes tomorrow? The answer generally was: you can export all your data and then you feed it into your local database. So the matter ended with: OK, we'll stick with our current database. Goodbye.
At the end of the day we saw such services as a glorified data-entry/store/search database that hardly will suffer many modifications or require updates, and since pretty much of our distributors (apart from mouser, digikey, avnet, etc.) won't support data exchange and PO/Offers will be negotiated in the traditional way (also negotiating price reductions). No need for VC funding or anything else.
We need to be able to open an schematic/BOM/whatever in 10 years and be able to operate as if it was just created yesterday, so much of the software we use is "a relic of the past" (in SaaS vendors eyes), but effective and "the way is supposed to be".
Online/cloud or subscription-based CADs? Also kicked out of the door.
With this I want to say that people like the article author seem to live in a little box, thinking on what they use daily as web-service-saas-whatever developers, without realizing the world is much bigger than they think it is.
It is pretty much sustainable. Except few natural cases SAAS is just a wet dream of vendors trying to have constant revenue stream the easy way.
If software needs support/new features you pay for upgrades. Or not if the old version works just fine for you. In my case I use tons of software for which I only paid once (sometimes I pay for upgrades as well). If I had to pay monthly fee for each title I use I'd be spending insane amount of money.
I strongly disagree. I have always, up to this day, sold software on that model, and I will probably always do so. There's nothing unsustainable about it.
What is true is that rent-seeking can be much more profitable. But fortunately for those of us who find it very distasteful, it's not necessary.
one-time fee only works if you have a constant stream of new customers into your niche that is sufficient to pay your salary.
with upgrades or subscriptions you can have a group of users who love what you do and are happy to support you without having to hope for new users and constantly market to get them.
If I want to continue active development on it, then I sell upgrades. If not, then I'll continue to do free maintenance releases, but my main business will be from different products.
> one-time fee only works if you have a constant stream of new customers into your niche that is sufficient to pay your salary.
You can continue to get income from existing customers. You just have to provide real value to them in exchange for it.
A good example are note taking apps. My notes should be private and I want to be able to read them ten years later. For me your product would need to add something valuable to this that a filesystem and a bunch of files that I synchronize myself can't do. As for now there is no note-taking app I found where the benifits outweight the preceived loss of privacy and reliability. The online thing can make sense, but syncing my phone with Nextcloud works even better, so I don't really see why I'd need it.
This is potentially different with an app like yours, because the benefits of using your app vs using e.g. a spreadsheet seem truly tangible. Using JSON and allowing export at any time is a huge plus. Having it a web-only kinda makes sense, as your app seems to be geared toward teams (and any serious parts managment makes a lot more sense when you are not alone).
While the additional work that would have to go into documentation and programming if you were to offer this as a self-hosted variant is non-trivial, from my standpoint offering the option to self-host can also be seen as an act of communicating: "Don't worry, whatever happens to this project, you will not lose the time invested".
I am not obsessed with this kind of reliability, but I just want to avoid the future hassle of having to deal with this, especially when I put a lot of time into it.
While you're right that doing this is less convenient, I can say with the personal experience of developing a few such projects on a shoestring that doing this isn't as hard or expensive as you're making out. It's entirely doable. It just requires thoughtful engineering.
I wonder how often the answer is simply “We know lots of JavaScript programmers and not a lot of Qt programmers.” Talk about the tail wagging the dog.
As a user, I strongly favor software I buy once, download, and run as-is, forever, untethered to the Internet.
However, that doesn't mean it needs to be "hosted" anywhere. It's trivial to make an app run a little web server that they access with their browser instead of a native or QT UI.
Browser based interfaces are an excellent bet (probably the best bet) for "run as-is, forever" or as close to "forever" as we can reasonably get right now.
I think you're not looking at this from a local-first perspective. From that perspective you can have the same app locally as on the server. There's only one version. Yes, it does require more planning and atypical approaches, but it's 100% doable.
I'm in the planning stage of a local first web-app that will have a server side version, and it's literally going to be the exact same code on both.
I can see some arguments for having _slight_ differences between server and client software but nothing that isn't easy for a solo-dev to maintain. Mostly set it up once and never touch it again type things.
thinking local-first has fundamentally changed some decisions i normally make without thinking for web apps, but I think it will absolutely be worth it for my users in the end.
And how do you plan to achieve that? What will be running on a server side? Will it be headless or server-rendered?
Do you have a GitHub repo with that project?
Sorry for lots of questions and thanks in advance!
It's basically a P2P based Dropbox with no accounts, full end-to-end encryption and no folder size limits.
It's not open source, but it can work without a central server if you need it to. It's also amazingly simple to set up, much simpler than sync thing.
if you pay for resilio there is an option to add all your folders in one go but on some computers I don't want to add all of them anyway so that's not much use to me. with the free version you have to manually add folders one by one but to do that you need the key which means you need to copy them to a text file and add them on another computer.
with syncthing, it will detect other syncthing devices on your home network so you just have to add the ones you want then accept the request from another device.
once that is done you select which folders the device has access to and then a notification will show up on said device asking you to connect. so basically no fiddling with keys or having to store them somewhere secure
(this is all presuming i was using resilio correctly, maybe there was an easier way I was not aware of)
https://github.com/micro/micro https://micro.mu/docs/network.html
Switching to another cloud provider this way is trivial and usually only involves changing the Terraform configuration to setup a k8s cluster on another cloud. All k8s-specific config/deploy files can be reused on the new cluster.
This of course only works if (as you suggest) you stay away from cloud-specific services (SQS, aurora, ECS, S3) and run everything in-cluster, or use managed services that are available on multiple providers (Postgresql via RDS, or Digital Ocean managed Postgres, Cloud SQL on GCP)
Based on my limited experience, I highly doubt this. Have you actually deployed cross-cloud k8s setups, or is this merely a theoretical statement on your part? Deploying to another cloud provider brings a whole new universe of failure modes and auth quirks, let alone migration and switch-over woos.
Note that most of these were not production clusters so switch-over was just data restore and DNS changes.
I build clusters from the start to not use cloud-specifics where possible and all cloud-specific configuration is on the cluster edges in terraform which you have to rewrite anyway when switching clouds.
Auth things like IAM permissions are not an issue if everything is “in cluster” and auth/permissions are checked there.
Most of these deployments consist of several application servers, PG databases, redis, rabbitmq etc
The cloud apps are not slow only because of moving data, but there is also a problem that an average server is fast(16cores CPU + 64GB RAM), but If it's used by let's say 100users, It means one user has only 0.16core + 0.64GB memory. So an average laptop(4cores/4GB) or phone(4cores/1GB) is way faster. Basically people buy billions of transistors to use them only as a terminal to the cloud. Not to mention privacy risks.
A week ago, I did showHN for skyalt.com. It's a local accessible database(+ analytics, which is coming soon). I'm still blown away how fast it is, that you can put tens of millions of rows to single table with many columns on consumer hardware and you don't pay for scale or attachments.
You have to consider the costs of building an app native for each platform vs a cross platform framework built with javascript. Single javascript code base for all platforms is a lot cheaper.
I am pretty sure that all of these giant companies releasing garbage Electron apps have a bigger budget than the budget on the 2 people (us!) that wrote what we wrote in 2.5 years.
When I see how abysmal apps like Skype and Slack are, I despair. Colossal amounts of RAM and simply displaying text and pictures.
It might be more convenient for the developer to write in the first language they learned but it is producing giant bloated applications. More CPU cycles, more allocations, more power consumed, shorter battery life on mobile devices (laptops), more charging of devices, more fossil fuels burned.
You care about delivering a high quality and high performance application, big corps just want to sell a product.
Also, just because they have a bigger budget doesn't mean they are all about spending it, they probably want to squeeze it as much as they can.
I can't quite get this point. From my perspective software engineers love/adore electron applications.
Look at VScode as the example:
- electron based javascript application
- telemetry included
- proprietary build with "open core"
It is literally the most popular code editor right now (p.s. I don't use it). Why as a tech savvy user you will use something you don't like for 5-10 hours each day to do your work?
Only answer I can see that electron is not an issue here.
That is, if JetBrains had made JS plugins first class citizens of their products, possibly VS Code wouldn't be as popular as it is.
Also, there's an expectation that an IDE will be somewhat heavyweight. I don't mind if VSCode or IntelliJ grabs a few gigs or RAM, because I live inside those applications and depend on the features they provide.
What I don't want or need is Yet Another Electron Based Markdown Editor that gobbles up half my laptop's memory so that I can edit a hundred line text document.
After taking time to learn my tools I can't use vscode anymore because of how inefficient and restrictive it is now for my workflow. I mainly prefer tools that can last a lifetime and time you've investend now can yeld much better results across several decades of usage.
Here is short list of tools I can't live without: magit, org-mode, undo-tree (actualy it was a feature request and vscode team said it's too compticated for broad audience), ability to hack your code editor as you wish. Ability to work all day without touching a mouse once.
I stopped using VS Code because of the billions of files it ships with (does anyone remember DLLs these days or what?) and its performance and debugging capabilities were very disappointing.
The people who build slow electron apps would have built slow native apps, if they could build one.
This is overly simplistic. You're pretending that cores/memory are "allocated" to users, but really, a user might only make a few tens of requests, and the server only needs to spend a second or two servicing each request. On a server with only 100 users, it could very well be the case that a user has all 16 cores + 64gb available at the time they make a request. Also, as another commenter pointed out, you could use a large chunk of that memory for shared resources, and then each request might only need a few mb of memory to service a request.
It also depends on _who_ owns the data. In an enterprise environment the company usually has a vital interest in the data and on-premise deployments are a good way of retaining cloud computing without giving up data ownership. I'm surprised that more SAAS products don't offer on-premise given the privacy and ownership benefits. The tricky part there is making software that is easy to deploy and maintain, which might be the reason that it isn't done more often.
A product like Grammarly that allowed on-premise deployment would side-step a lot of the issues with sending all that data to a third party. I can't imagine a law firm ever being able to (legally) sign up for that.