Ask HN: Junior dev in charge of rewriting 500k line PHP app. Looking for advice
We are primarily a CRUD app, but we also integrate with external APIs (e.g. to handle texting) and utilize Amazon S3 and SQS for generating reports and other long running tasks. Tentatively, I want to propose a Model-View-Controller-Services architecture, where each model is a thin database abstraction layer that knows nothing else; views are pure and idempotent (à la React components, but server side); each controller is responsible for one endpoint, receives a Request object, delegates work to services, and returns a Response object; and most work happens in services, which can call each other and models. We delegate authentication to Google, but will need to implement a very fine grained permissions system. I want to keep things simple, and avoid bringing in too many dependencies, so I am leaning towards a minimal set of Symfony components, rather than something like Laravel. One of the primary complaints we get is that our current system is too slow. In part, this is because most actions trigger a full page reload. I want to use HTMX to increase responsiveness while still keeping most of the functionality in the backend. We will also be using Docker. We may use Redis for session management, or we may again store session data in the database. Our application will be deployed on DigitalOcean and use Cloudflare for caching.
We've agreed to have one large component of our app ready by Apr 1st, which our largest client will use while we finish the rest of the work. Our development strategy will be based around delivering this key component, plus the minimum number of features required to support it (accounts, permissions, ...). I am curious if anyone could share their experience building a new system from scratch, and what pitfalls we might try to avoid while focusing on delivering this component.
Our data is relational in nature, although there are some areas where we will need to store JSON/markup (we allow users to create custom forms, workflows, and templates). We will likely use MySQL 8. One of our founders wants to have a separate database and deployment per client, because he is concerned about accidentally showing Company 1's data to Company 2. I think this may be overkill, as some of our clients are very small, but also because it adds developer overhead needing to make changes to X databases and X deployments, rather than 1 shared database and 1 shared deployment. (One problem we currently face is that updating/deploying Star is manual and time consuming.) Right now, we average on the order of 100s of concurrent users, while hoping to grow to 1000s of concurrent users soon. We have users across the US & Canada, but are unlikely to expand beyond those markets. Is a shared database and single deployment reasonable for us, or should each client have their own database?
26 comments
[ 4.7 ms ] story [ 74.3 ms ] threadOn dev box, test out/validate assumptions of various modules / interactions noted above.
Should be able to get a feel for code & how being used (internally & externally) for ideas on how to approach end question.
It's a rare project that a rebuild succeeds!
My experience is with Laravel, so not too far off your ideal of using Symfony, though I wouldn't let minimalism cause you to avoid Laravel and end up having to do more work. Whether you use Laravel or Symfony alone is probably negligible compared to the application itself.
For such a large application - in keeping with avoiding rebuilds - I'd be looking at ways to subsume parts of the current application into (say) a Laravel application, with glue code at the edges where necessary to paper over differences between the legacy code and the framework-backed code.
Continuing on that path with parts of the system - while (crucially) keeping it up and running for existing users - slowly replacing/enhancing existing code to work in a more coherent fashion with the guide ropes of a framework.
Eventually you ought to end up in a position where you can pick up speed and tap away at the legacy parts.
Ref: your final paragraph, in a decent framework you can quite easily scope queries to restrict the data queried via SQL. Particularly if you have a strong permission system this becomes easier! I like to avoid database-per-customer unless absolutely necessary, it has its own issues, like keeping all databases in sync and on the latest migrations.
Sounds like you've got a fun job ahead of you, holler if your org could use some help :) UK based.
Rewrites don't go how we think they will, and the cut-off and migration between the two certainly doesn't either. It becomes unwieldy very fast, and in ways we couldn't predict.
I've had to do a handful of very large PHP-based rewrites, and I would still recommend a piece-meal approach. It isn't so much that one way is inherently "better" than the other as far as the code goes; git is amazing, and tests do what tests do. It's more that you can reason about the features, changes, and data flow if you take it one piece at a time. And also, lots of tools, libraries, and services that you use will release updates in the time that you're rewriting, which causes its own headache too.
Sure, the tradeoff is that going one piece at a time takes longer, but you have time on your side because you have revenue and a a customer base. If you were a startup that didn't have that, sure, go for the fast approach and fix it later.
But I think when all is said and done, we can go fast and build something that doesn't work, or we can go slow and keep both things running and swap out smaller pieces along the way. As they say, "slow is smooth, and smooth is fast."
Also, holler if your org could use some help. Seattle based.
I've seen similar in everything from MS Visual Studio to Notepad++, but with PHPStorm it's taken even further and works much better. Really helps with refactoring, since I can do things like get an overview of an API, see what stands out, and "smart rename" methods - which then fixes it throughout the code.
I second everyone else's opinion of "this is too big to rewrite all at once". Break it up piecemeal and replace one feature/area/service/whatever at a time. Your app sounds way too big to be able to successfully rewrite all at once without breaking edge cases and pissing off customers.
Now to venture into IMHO territory... this is my biased opinion as someone who grew up in PHP (including Laravel/Symfony/Drupal/Wordpress/Twig/etc) and eventually moved to fully Javascript clientside apps.
I think the MVC paradigm works well for environments where your app can inherently be a full-stack monolith, like a desktop app. But it can slow down work in mixed environments like the web where the frontend world moves much much faster than the backend, and is usually made by entirely different vendors (Google, Apple, Meta, Vercel) than the backend (all the FOSS stuff, plus Redis and VMs, etc.). It makes it so that it's really hard to change any one part of the stack without changing or destroying the rest, which means your frontend can only move as fast as your backend.
This might be a situation where decoupling the frontend and backend more might help make it all more modular, i.e. the web UI just calls APIs, as opposed to PHP controlling everything from the server side and rendering changes there. That separation can help you with both a modular rewrite and also future changes. Your backend won't have to be as volatile as the frontend. Basic CRUD stuff hasn't changed all that much and PHP can handle that just fine, but I think it's a bad idea in 2024 to limit yourself upfront to Symfony and HTMX, both because you're tightly coupling your presentation to your data/API layer and because probably by the time you finish the rewrite, the web frontend world will have moved on even more. Both of those are already quite niche compared to the big JS systems, so if your goal is increased velocity, using those systems would make it harder to hire and your codebase harder to maintain. It's also harder to scale with # of users if all your services and rendering for all your customers live in the same VM. And it makes it really hard to shard your services across customers (if you ever did want to move to isolated databases, which is a big if).
On the UI performance side, really it all boils down to using more AJAX, which HTMX and PHP can both do, or you can do it manually. But that works better if your frontend is decoupled from your backend, so the server isn't tied up trying to render new HTML chunks while it's also performing database lookups and mutations. The decoupling allows you to more easily manage complex frontend loading/error/caching/optimistic update/network error/etc states, and more cleanly dictate which UI updates can happen entirely in the browser and which to wait for async server data. It also saves server resources since you're not rendering any HTML for the client. Phones and browsers are plenty fast for that these days, and it makes the end-user UI much snappier, usually.
Even if the network request is a fixed value (i.e. the network roundtrip time is the same regardless of your frontend framework), being able to do dynamic clientside DOM mutations like that will make it feel faster for your visitors, even if your backend stays exactly the same. It also makes clientside compone...
2. Use the Strangler Fig Pattern (aka strangler pattern) to slowly rewrite the app in small iterations with a feedback loop. Write a new component that slowly takes over the old component and brokers between the two until the old one is, well, choked out. You state you're running docker, so figure out what the new tech stack is, call it new_stack (the tech choice here doesn't really matter for a web app and this pattern):
- deploy a docker container running nginx, and new_stack
- deploy a second docker container with the old PHP stack on it (old_stack)
- setup nginx on new_stack to proxy all HTTP requests to both containers
- at first it proxies everything to old_stack container
- rewrite some full service part of app in new_stack and change the proxy rules to route traffic to new_stack for that service
- repeat the last step for each part of the app as its rewritten
- if you run into major issues, update the nginx rewrite rules to route back to old_stack.
DigitalOcean and CloudFlare probably have features to do what I describe above, I don't have details but look into load balancers where you can shard traffic to different servers based on the HTTP route.
But let's say that the business itself is doing good and there truly is a need to improve the tech now to help do even better. In that case, DO NOT DO A FULL RE-WRITE. DO NOT. sorry for caps but it is important.
Like many others have already suggested, start with a smaller piece and improve it. May be create a separate API/service. Then measure the improvement in developer time and customer happiness linked to it. Build that slowly over time.
"I want to keep things simple, and avoid bringing in too many dependencies, so I am leaning towards a minimal set of Symfony components, rather than something like Laravel"
That is good thinking.
"One of the primary complaints we get is that our current system is too slow. In part, this is because most actions trigger a full page reload."
Instead of a full rewrite, focus on improving te performance here. Yes, a full rewrite sounds exciting but how will that impact the business ? Instead, what if you figured out a way to improve the current performance bottleneck first ?
Remember that customer don't care about tech. They care about a solution that solvs their problems fast, efficiently at a reasonable price.
I am also concerned about the fact that a junior dev is being asked to rewrite. I understand it is just 4 of you but depending on the number of customers you serve, do not take this lightly.
Source: I have built software and run my own SAAS. 20+ years of experience breaking shit in production.
If OP is basically soloing a rewrite of an 11 year old company, they seriously need to consider asking for a new title
With no offense to you, a junior dev leading a rewrite of a large messy application is almost guaranteed to result in a different messy application. That's really the most likely result regardless of who is doing it.
I would add on to what everyone else is saying and say don't do a full rewrite. Instead, refactor small chunks at a time.
> One of the primary complaints we get is that our current system is too slow. In part, this is because most actions trigger a full page reload.
Full page reloads don't really need to be slow. Measuring and improving web page speed is well established, and IMHO you should work on this and understand it before you build something new; there are a lot of ways to build a page that will be difficult to make fast, having experience making things fast will help you start on the right foot towards fast pages.
You should be measuring at least time to first byte, time to last byte, and data size on the server; times in ms. On the client side, assuming you've got client side analytics, you want to measure the kinds of things that Google PageSpeed Insights measures: https://developers.google.com/speed/docs/insights/v5/about If you can get this from (possibly sampled) real clients, that's best; but if not, at least test occasionally with the tools inside your browsers.
A very fast PHP page can be done in < 10 ms, server measured. But more realistic is time to first byte < 15 ms, full page content in 50-100ms. If you're much beyond that, you need to measure parts until you find the time, and fix it. You can work this problem at the same time as some refactoring by including timing with refactoring; chances are, a lot of the server time is in database queries, and instrumenting timing on database queries and centralizing the database interface into a single place can go well together. Ideally, you can push this timing information into your access logs, and you'd have a list of timings in each log line; time to first byte and total request time are best measured by your webserver rather than in PHP itself; especially if you're using frameworks that do a lot of initialization on each request, you must include that in your total, but it can be difficult to measure from within PHP.
Since your users are all in US/CA (and I assume your server is too), if you get your time to first byte under 15 ms on the server, you should see it under 100 ms on the client. And if your pages aren't too big and the dom isn't too complex, a 100 ms server page time should be at your clients within 0.5 and rendered in < 1 s. You can probably do better with partial page updates, but see what you can do with making full page updates fast first.
It sounds like the industry changed. Get business processes or business data requirements first. What do they want to keep, change, get rid of, or add?
Template out your business objects and data structures. On the topic of a single DB... I wouldn't. Unless your clients are sharing data, that's just a huge security risk to keep it all together with website accessible from anywhere.
Then make your changes gradually. I assume you have no tests to look at for requirements, otherwise that would be helpful. It might be possible to migrate some pages or functionality by pulling in the new stuff using some sort of hybrid routing or microfrontend.
Any way you look at this, these sorts of rewrites are painful.
You can select a release date. Or you can select a release feature set. You can't select both. Since you've already got a feature set for your primary client and a release date, you're already under time pressure. That doesn't bode well for success.
Time pressure means you optimize for "done" rather than "done right". Which means you cut corners, will "come back and fix that later" (which never happens) and in 5 years you're back where you started.
I rewrote a big DOS system in Windows and it went well. Primarily because; A) there was no "backwards compatible" goal. (We could import user data, but configuration was very different, and not portable.) B) the -experience- of working on the dos system had given me insight as to where the bottle necks and problems were. So the -design- looked at those problems and made them go away.
That program is still going strong 25 years later (and has been under continual development all that time.) Because the foundational design is right, it's been added to coherently and none of it is too-brittle-to-work-on.
My point is that this is a "generational change". Getting the design right will be worth years of revenue at the other end.
Your bosses domain knowledge is key. Coupled with their experience of what works well, and what works badly (code no one wants to touch, regular support issues, features that can't be implemented etc.)
90% of my regrets are around data design. We used numbers as primary keys (pretty normal for the 90s as space, ram and cpu were limited). But that has made some tasks much harder now (database merges, distributed data etc). Today I'm a strong advocate for UUIDs (especially UUID7).
Personally I like the different-database-per-customer approach. It's better for security, and allows us to copy production data for support and development. It also scales better. (Ie horizontal scaling).
Since the original program ran local, each client had their own database anyway. We mitigated upgrading hassles (a BIG issue in the DOS system) by making the program itself upgrade the database. (It checks, and runs upgrade tasks on startup.)
One big advantage of this approach is that we can roll out upgrades slowly (one client at a time) when we're doing big changes, and upgrading one client doesn't bring down others.
But those are problems and solutions in my space. They may not be appropriate for your context. Each design decision is a trade-off, understanding the upside and the downside of each one in the long-term is key to making the right choice for your context.
Sometimes hedging your bets pays off too. For example you can populate your database with tenentID fields. You can code as if for multi-tenent. But you can deploy as one tenant per database. That approach gives you flexibility later on.
Good luck. It sounds like a fun project. But this foundation will ultimately determine how long this generation of the software is good for. Don't rush it. Gather as much information from your bosses as possible. Present design options and implications - but ultimately get their buy in to choices made. They need to understand the pros and cons because you'll inevitably hit a con one day, and it's best if they understand its a decision they made, not you.
1. First, get the database design right, simplify it as much as possible, and consider that you are going to migrate data from the old design to the new design too. Chances are that some of the old structure might not be needed anymore. Work with your client to determine which tables or fields you can already remove. Also, for data that is not expected to be frequently searched or retrieved through complex queries, considering organizing those in JSON format.
2. Do not code right away. Do a very thorough plan of all the controllers and models you need, and review functions that will be reusable, especially in the models; do the same thing for helpers. You mentioned that it was built by a founder without formal training. There’s a good chance a big part of the code does not follow SOLID principles. Your ultimate goal in this planning is to reduce duplicate functions and make it easy to write test cases for your code, both in the BE and FE.
3. For complex or big features, regardless of your team’s experience, I would encourage you to still use flowcharts. Finalize the logic first before coding. You may find logics that can be simplified or is critical to your database design if you do this.
4. Implement TDD, then configure your CI pipeline to automatically trigger test cases and block merging if a single test case fails. This will instill discipline in your team and reduce the number of bugs.
I don't know if it's an option for you to switch to another language. If it's an option, use TypeScript. Strict typing helps early detection of errors caused by incorrect types or accessing missing properties.
Before rewriting the whole thing, are you SURE that you can't make that thing faster?
Speeding things up is not too difficult usually. My experience is that there is always something that takes up 95% of the time. And if you find a way to make your app a lot faster without rewriting it, you'll be a hero.
Replacing the app sounds painful. And btw the architecture you propose sounds way too complicated, the 10k-line controllers are more manageable than your microservice proposal.
Hear hear! This is _very_ often true.
You can profile your database from the command line, looking for query optimizations. Usually, adding some indexes and removing "useless" joins does the trick. Maybe you can convert schema to take advantage of specific optimizations? Maybe if you're storing blobs you can move those to the file system?
You can check network usage in the client, maybe the app is loading things it does not need? Maybe the JS stalls waiting for a CDN request? Maybe images, if any, can be optimized and/or removed?
You can look into the server process list, and check if its hitting the disk too much, if there are any permission issues, if there's enough RAM, whether the database is local or networked (databases can peg your CPU, which can be costly if its on the same machine -- i know its not recommended but I've seen a lot of weird stuff).
> And btw the architecture you propose sounds way too complicated, the 10k-line controllers are more manageable than your microservice proposal.
This is also often true, but a hard pill to swallow.
Good luck!
1. This will last years and never be finished
2. You're not starting from scratch. Not sure why you want advice on that. You're building something that needs to integrate with an existing pile of garbage. Most probably it'll be an absolute ball ache especially around data management.
3. Because of the above I'd just leave. Seriously. It'll be a mammoth task and if all you're getting is a junior salary it's probably not worth it
4. Just use laravel and architect things the laravel way. Using services as well MVC is fine, especially if you'll have an API too, but in general the less you diverge from the framework's usual approach, the more maintainable it'll be.
5. Of course there's a risk with a single DB. However if you architect for a single DB, there's nothing stopping you creating a dedicated one for critical clients.
6. Have you thought about leaving? Expect this to take several years and dominate all dev work until then (or lose steam at some point and be stuck with a mix of both). Ultimately this work adds little to no tangible value to customers and at some point management are likely to want to see new features being added again.