We had to turn off a bunch of on-by-default features in the system I'm deploying now because IOS matches too aggressively and decides that arbitrary UDP datagrams are DHCP requests or PTP packets that need rewriting (and then dropping when they turn out to not be DHCP/PTP). I have to think that an AI rewrite with a today-good model would recognize the lax checks and do them better.
It’s been a while, but configuring their hardware manually in 2016 was bug ridden ridiculousness. Awful UI/UX. Their own techs had specific field notebooks for our make and firmware version that we couldn’t access unless persuaded, detailing various land mines to avoid while working. Literal “don’t click these things in this order” kinds of notes.
A critical security firmware update once brought down redundant pair of firewalls we used because the bug didn’t manifest until a counter buffer overflowed months after application. Three hours into the outage the tech power cycles the thing and our services came back to life.
Cisco is only doing this because some other alternative is more expensive. They had more than a decade to get started.
Sometimes old code comes with its own problems that are not the code but the ecosystem they depend on or run in.
E.g. some critical security vulnerability that is not backported to the library/compiler/runtime you are using in production, or you can't get spare parts for your servers any more so need to upgrade to something newer but you can't run your old OS version on new hardware so you have to upgrade the OS too and now it's dropped support for foo etc etc. Lots of things pile up.
As much as I dislike AI, having worked on Cisco gear for years, I can't imagine AI can put out a worse product. If the re-write is full of bugs and security holes and static SSH keys, then it has not changed at all.
The problem with rewrites are that even if the rewrites are ultimately good it will tend to first create bugs on an existing thing that are presently churning in a stable state (well or not).
It doesn't, but Cisco was great and they've just fell to a laughing stock. Their stuff is really bad these days.
And they've overstretched their product line to too many things, trying to be the "be all" company. Fortinet is making the same mistake, but they're earlier in that process.
I mean, it's not really a mistake on their parts, their primary goal is to get rich and they are succeeding.
Citation: I've been working with Cisco technologies since the mid-1990s. At my current job, we are a few weeks away from phasing out the last of our Cisco junk. And since people will ask: to Juniper/Mist and Arista.
Or perhaps the lack thereof if you are one of the unsuccessful companies. In the 90s, I worked for a company that competed with Cisco I worked for a company that competed with Cisco. story was in the mid 80s an executive went to Silicon Valley. His report was that Cisco was a bunch of nobodies that would never go anywhere, which they'd go down the street and buy Vitalink instead. You have heard of Cisco and not Vitalink for a reason.
This is very true, and it's easy to forget that it often has nothing to do with how good/great the product is, unless it is self-evident and thus self-promoting (which for most non-trivial things is simply not the case).
Cisco did provide updates via CCO and anonymous ftp server. They quite openly told that just download new image and test in your lab, and then if feature set did require licensing, please do it before deploying production. Trust was put on customers and they did very little to check and verify if customers did comply. Which is quite contrary to what for example Oracle practise is been.
>If history has taught us anything. It’s that the measure of a successful product is usually more to do with its sales people.
As someone who worked Cisco gear over thirty years I still remember why us networking did want cisco instead of many other brand devices and it hadn't anything to do with marketing and sales people did.
The Cisco IOS text based configuration was in early days of internet which made it so popular.
Many other manufacturers like HP, 3Com, Cabletron, Madge (IBM), Bridge, etc. did not get why we wanted and complained “give us textual configuration format like IOS has”.
With a plain text based configuration it was easy to adapt configuration from one model to have ready made templates for each model, upgrades and migration to new models were easier, you could use version controls (rcs,cvs, svn,... long before git) and then when RANCiD came it was easy to automate config backups, revert and rollback changes, pass audit with human readable configs and what was in production even long time after etc. In short text based configs made it possible to build large network scale configurations management tools.
IOS has compact, but quite easy to read config snippets made it much easier emailing others, posting examples newsgroups, opening TAC at Cisco CCO, books with examples, learning and teaching etc.
It took quite long before for example HP understood and switched text formats right after turn of century. Juniper seemed to understood right from beginning. But some like Fore, Nortel, Alcatel (ATM-switch makers) and many others did not understand before going out of business. Most of these others had either menus (3Com, HP) or a lousy CLI with obscure unique for just that device model and quite often not even option to print config or if they had not just with changes from default config but printing everything and providing no means to drop showing default. This made very made it manually while online very hard without external version control to even know what is current status and what had been changes.
BTW, this is still an issue with some manufacturers haven't understood and makes their devices much worse than they understand. Fortigate firewalls CLI is still like this. You can select a subsection of config to print, but it will no indication what differs from default configs. And the whole firewall config is hundreds of kilobytes. Good luck finding what's been changed unless you had saved default config when you started using that device!
When you have thousands to even tens of thousands of network devices in use as many were working you need to be rely that even when you encounter device failure and you don’t have same model any more available, you are able to adapt and find working fix configuring something else doing what is needed.
Binary and any non textual configurations may fit the purpose SOHO category devices, but when network grows even bit larger and configurations more complex there isn’t anything better than text base configs as far as us humans will need to configure and maintain those devices.
Cisco have had issues with software quality for sure, but it was Cisco IOS that was the reason it became so popular and we wanted to use it.
Juniper when it came end of 90's did understand this and made even better configuration system for Junos than IOS was and still is. Cisco have improved things with IOS-XR, but it's still not up to what Junos is currently.
> Juniper when it came end of 90's did understand this and made even better configuration system for Junos than IOS was and still is.
"even better" is an understatement ! I was working on a dual-vendor network at the time, we ran Cisco and Juniper in parallel.
Working on Cisco always just felt like going back to the 18th century, I mean the whole candidate and active config thing alone in Junos was insanely valuable:
show | compare # diff the config
commit check # lint it
commit confirmed 10 # apply it, but don't persist unless confirmed
commit # confirm
Meanwhile, back in 18th century iOS land, a fat finger on the ACL would leave you with a bad day in the office until you rebooted the box via OOB.
I’ve taken entire systems offline because of unexpected command expansion.
My “favourite” was “admin”, which was used as the login user, was also short form for “administer-all-port-shutdown” (or words to that effect; it was a long time ago). So if you were distracted when logging in, you could easily soft-lock the entire device meaning you’d have to then run into the DC, find the hardware, and physically power cycle it.
I think that juniper learnt from Cisco's mistakes while Cisco is stuck with their config command structure as they decided to chase backward compatibility and familiarity across products.
Agreed completely. Junos have niceties like copying parts of config, adding config even before cards interface cards are inserted, renaming/moving parts of config, of course quickly config parts, viewing and exporting config various ways etc. etc. Cisco improved config viewing a bit with IOS-XR too, so that it's possible to scroll like Junos had ages ago.
However there are also things Junos vs Cisco was that Cisco had made bit easier. Just to mention few
1) a simple and quick way to add aliases to config file, which made life easier with long commands that could be otherwise always in hurry hard to remember. Now later versions Junos they have made this better, before it involved twiddling with XML files OS-level etc.
2) IOS TCL scripting, that proved to be quite useful automation tool.
3) OS upgrades of course with a single flash blob made it easier than upgrading or downgrading FreeBSD based Junos.
I’ve used Cisco for decades too. And the competition caught up with Cisco a long time ago. These days Cisco is trading on their brand (and even there, it’s often not enough).
The last large scale networking project I did was more than 10 years ago and we went with HP.
Personally I don’t want direct console access. I want a modern DevOps pipeline with version controlled config. I’ve got too many battle scares from accidental typos in IOS. I have no intention or reliving that error prone way of network management again.
Back then they only really did routing and switching, and those are almost the only things Cisco does well.
Every new product of theirs I’ve touched in the last 20 years has been trash, and even the R&S golden goose is being dragged in that direction with customer-hostile functionality like Smart Licensing.
"Cisco Access Points That Are Running, Or Have Run, Releases 17.12.4, 17.12.5, 17.12.6, or 17.12.6a May Exhaust Flash Space, Preventing Upgrades"
Recovery path basically involves either factory resetting every single AP or up to, if it's already bricked, Cisco TAC + you having to console into every single bricked AP you have to pass a secret challenge/response at the bootloader to flash a recovery image :)
You don’t have very good imagination then. Cisco code is such a mess that there’s basically 0 chance he can even make a harness in 24 months, much less rewrite production code.
So there's a real risk that Cisco might become history in just a few years. Taking that kind of risk with a rather unproven technology has to be psychosis. Or is that kind of belief a form of Religion?
Does this one engineer have the domain knowledge to know WHY the old code was written the way it was, and what the graybeards would have done differently if they could start again now?
Otherwise he's just going to replicate all the legacy bugs and questionable design decisions.
Looks like his startup was some kind of translation service. So yeah he is totally qualified. It's a miracle how Jeetu Patel was able to find Padmanabhan Krishnamurthy or even know that he was under employed languishing somewhere in the bowels of CISCO. I wonder if they know each other from India?
I've never heard of Ponytail before, I've read the entire README and glanced through this repo and I'm left so confused. From what I can gather this project is essentially a single markdown file with a prompt that verbosely says "write the simplest code". Yet the repo itself looks so bloated and complex and has so much activity. This feels like satire similar to fizzbuzz enterprise edition[1] but apparently this is the actual state of our industry?
It's a bunch of text files in a repo with massive churn (all new version 5!), but somehow even the technical debt 'skill' (i.e. text file) has technical debt itself:
Is this like a PR piece for the guy? It's odd to see him singled out in a piece that would be normally be about the company, the CEO or the decision itself.
The racism is blinding. Cisco acquired his startup which was YC funded (which you placate you because it's 'white' approved). So technically, even by your own rules/biases this kid is smart.
We have an embedded system in one of product offerings that was largely developed using a bunch of “antiquated” technologies (over the past twenty years) and over the past year the two software devs that maintained it have retired. This normally would have big problem, but the entire codebase was re-written, documented, and modernized in a matter of about 3 months (for about $20k in tokens, and a mid level developer spending 50% of their time on it over that period). So it doesn’t surprise me to think a company like Cisco has piles of situations like this.
Every time something like this has been done on my company I follow up with the question: did anyone watched the code to see if the business logic is the same between the two versions?
The answer is: but but but automated testing.
Followup question: did anyone read the automated test suite?
All software has bugs, but right now you have a legacy product in production that works, then you have a rewrite that nobody tested. Do you accepted the risk to yolo it?
Why do people doing a rewrite need to have read the tests?
If you have good test coverage, and the tests are reliable, then what's the problem?
If your engineering culture is bad already and you don't have good test coverage or they're flaky or whatever then that is different. If that is the case, it is probably from smart-ass attitudes like yours asking these "clever" rhetorical questions to derail people trying to improve things that led it to be like that. Do everyone a favour and keep your mouth shut and let people fix things.
I guess Cisco decided it has not alienated its customers enough till now & its finally time to get people running for the woods - at least if those peoples would like their network to continue working.
Indians have always been stereotyped for sycophancy and slavish adherance to spec over common sense, so it makes sense that that would go with being AI pilled.
I don't want to sound racist (which I'm not), but in my experience Indian software developers have often been less careful and concerned about quality than the average developer; it probably comes from many only becoming so for the job prospects, and not having much computer experience before college.
So it's not surprising if they embrace AI.
There's also that Indian cheaper outsourcing companies risk being displaced by AI, if they don't adopt it extensively to further reduce their pricing.
The engineer is the former founder of a company acquired by Cisco, so there is reason to believe they are exceptional and possibly more experienced and mature than most others at the same age.
27 is still quite young though and very possibly over confident.
I think we can all agree that the task as set forth in the PR puff piece is impossible. That said however, current AI tools are amazing for cleaning up and reanimating old codebases. I've been using Claude on a very large Java protocol proxy product and it's been able to do amazingly useful things for me. I originally wrote much of the code but Claude now knows much more about it than I do today, because people forget stuff over decades. It will sometimes remind me "as you noted in November 2016 this feature was designed to...", based on its analysis of the code and git commit messages.
Well they say the last 20% of the work takes 80% of the time so if he gets the prototype in 4 weeks, which I have no doubt he will be able to do, and assuming that prototype is the first 80% of work, then he should be able to do this in 20 weeks. He even has some margin! Looking forward to seeing the results
I mean, this is going to be easy. I've worked on Cisco gear before and this will be a simple job for AI. The big question is what the goal is... so you've rewritten new C++ code or ported to a newer version of Linux? Or you've unified the codebase of the Catalyst + Meraki switches?
Not clear to me what he's doing. Re-implement switching? Simplify features? Reduce LoC? Add test coverage? Either way, AI in that world will be helpful. I'm sure it'll spot all kinds of problems in opaque drivers that are hard to follow from MF specs.
'98% of the company's legacy code' is very vague. How many code bases? How each code base is connected to any supported Cisco product? How many KLOC the company's legacy code?
These are good questions that we won’t get answers to anytime soon.
I’d love to know what internal services and tools that are included in this, as well as any deliverables that shipped as software products to customers.
If it’s legacy code for IOS firmware, however, I’d be very worried.
It's way more complicated than that. Bear in mind many of the network protocols in use today are decades old and have been constantly evolving, but there has generally been a consideration to maintain interoperability.
Then consider that not all vendors implement the standards identically, because the standards can be a bit woolly and in some cases followed rather than lead live implementations.
There will be a bunch of code that looks useless, but is actually there because of specific interoperability scenarios.
I'm probably not describing the scale of the issue adequately. Lots of these bugs only show up when you have a specific bit of hardware on a specific version of code, and it's connected to another specific bit of hardware running another specific version of code.
Lots of this stuff should work in theory, you only find out that it doesn't in practice when you try it on real hardware. And there are thousands of bits of hardware.
Indeed. Making a perfect clone of such software is I suspect provably impossible because you have no way to assemble all the test cases. There might be some peer box out in the wild somewhere that is the only thing that will exercise some code path. You have no idea what that box is or what it does.
you assume their code is well written, while in reality the history of weird quirks and thousands upon thousands of CVE doesn't make the code underneath look well designed and written.
But yes, fact it uses a bunch of very well documented standards with multiple implementations gonna help massively
I suspect we are approaching a world where this makes sense to do annually. Opus 5.5 is good enough to improve on most codebases, and Opus 7 will no doubt be good enough to improve on that.
Current models are amazing but production code incorporates unknowable information through decades of diagnosis and bug fixing (and sometimes the opposite: working around bugs in other software that it talks to). Put even the most amazing coder that was ever born onto improving a complex old codebase, and you can guarantee they are going to break stuff. AI is better but not infinitely better.
People keep repeating this but on my experience is also very good at finding kirks of specific compiler versions.
I think it is a reasonable risk to encounter these bugs again in order to get a more clean and documented codebase. Especially when asking AI to plan and A/B release is also trivial.
> “I’ve given him unlimited tokens because he’s going to refactor code,” Patel told Fortune at a small roundtable dinner. “So I’m like, you can go, you can go nuts.”
I think that Fortune story is better than the Tom's Hardware one. It ties the Cisco anecdote into a larger trend of some people deliberately hiring younger, less experienced developers for their AI projects - because they are more likely to be "AI native".
Sadly, I think the label "AI naïve" is better applied to the many experienced professionals who refuse to engage with AI systems as a matter of principle - or who tried out ChatGPT in 2024, saw it make howlingly obvious errors, and wrote off the entire category.
I frequently find myself in conversations with people who are resistant to AI and turn out to have a very skewed idea of what it can and cannot do.
> or who tried out ChatGPT in 2024, saw it make howlingly obvious errors, and wrote off the entire category.
Or they were reading your comments and articles and couldn't reproduce any of the fantastic results that you were claiming, even back then. Things have indeed changed since the first Opus release about a year ago and LLMs are good at programming now. But still, experienced professionals are not so keen to sign off 1000s of lines of code that they never read, take responsibility for it etc. Also, it is a different job and it will take some time to adjust. Can you blame them?
Moreover, other trades have their own guilds or are otherwise protected through regulation. Can you blame programmers for fighting to keep doing things as they were doing it until recently?
Given that I've shared almost every prompt I've used since I started publishing about this stuff over four years ago I think I've given people a better chance of replicating my results than pretty much anyone else writing about this space.
What did you have trouble replicating?
> Also, it is a different job and it will take some time to adjust. Can you blame them?
Not in the slightest. This stuff is really hard. The changes it makes to our production are disruptive and, in many cases, unwelcome.
Yes, I’m sure the Indian man at Cisco acquired a translation company founded by another Indian man because he’s young and AI native, not for any other reason.
He'll pull it off, get feature complete, and spend a year or two with better models as they are released ironing it out. We just did an experiment, in four weeks generated 3500 commits, 200k sloc with a 1:1.15 ratio of production to test code and it all works very well. We are well on our way from 10x to 100x.
It is for internal use and so far so good. We are finding bugs and we just point the models at it with a strict red-first TDD approach and they've done very well fixing them. It certainly isn't perfect but the issues are very similar to what you get with a team of humans doing the same thing.
The main takeaway though is we had one person, in a single month, do the equivalent of at least a couple years of work for a small team. We view bugs and performance issues as a currently inevitable part of the process and we are dealing with them with models just like we do with humans on every other project already. Overall though it is representing an absolutely massive speedup. If you work in an environment where the models make egregiously more mistakes than your human counterparts then I am quite jealous.
ha ha .. that sounds like an Onion article. To be fair - if they did a narrow refactor to focus on fixing only obvious and known issues, they probably might be able to pull it off because a ton of that code is hairballs.
223 comments
[ 0.84 ms ] story [ 108 ms ] threadWe had to turn off a bunch of on-by-default features in the system I'm deploying now because IOS matches too aggressively and decides that arbitrary UDP datagrams are DHCP requests or PTP packets that need rewriting (and then dropping when they turn out to not be DHCP/PTP). I have to think that an AI rewrite with a today-good model would recognize the lax checks and do them better.
A critical security firmware update once brought down redundant pair of firewalls we used because the bug didn’t manifest until a counter buffer overflowed months after application. Three hours into the outage the tech power cycles the thing and our services came back to life.
Cisco is only doing this because some other alternative is more expensive. They had more than a decade to get started.
E.g. some critical security vulnerability that is not backported to the library/compiler/runtime you are using in production, or you can't get spare parts for your servers any more so need to upgrade to something newer but you can't run your old OS version on new hardware so you have to upgrade the OS too and now it's dropped support for foo etc etc. Lots of things pile up.
And they've overstretched their product line to too many things, trying to be the "be all" company. Fortinet is making the same mistake, but they're earlier in that process.
I mean, it's not really a mistake on their parts, their primary goal is to get rich and they are succeeding.
Citation: I've been working with Cisco technologies since the mid-1990s. At my current job, we are a few weeks away from phasing out the last of our Cisco junk. And since people will ask: to Juniper/Mist and Arista.
If history has taught us anything. It’s that the measure of a successful product is usually more to do with its sales people.
Cisco did provide updates via CCO and anonymous ftp server. They quite openly told that just download new image and test in your lab, and then if feature set did require licensing, please do it before deploying production. Trust was put on customers and they did very little to check and verify if customers did comply. Which is quite contrary to what for example Oracle practise is been.
>If history has taught us anything. It’s that the measure of a successful product is usually more to do with its sales people.
As someone who worked Cisco gear over thirty years I still remember why us networking did want cisco instead of many other brand devices and it hadn't anything to do with marketing and sales people did.
The Cisco IOS text based configuration was in early days of internet which made it so popular.
Many other manufacturers like HP, 3Com, Cabletron, Madge (IBM), Bridge, etc. did not get why we wanted and complained “give us textual configuration format like IOS has”.
With a plain text based configuration it was easy to adapt configuration from one model to have ready made templates for each model, upgrades and migration to new models were easier, you could use version controls (rcs,cvs, svn,... long before git) and then when RANCiD came it was easy to automate config backups, revert and rollback changes, pass audit with human readable configs and what was in production even long time after etc. In short text based configs made it possible to build large network scale configurations management tools.
IOS has compact, but quite easy to read config snippets made it much easier emailing others, posting examples newsgroups, opening TAC at Cisco CCO, books with examples, learning and teaching etc.
It took quite long before for example HP understood and switched text formats right after turn of century. Juniper seemed to understood right from beginning. But some like Fore, Nortel, Alcatel (ATM-switch makers) and many others did not understand before going out of business. Most of these others had either menus (3Com, HP) or a lousy CLI with obscure unique for just that device model and quite often not even option to print config or if they had not just with changes from default config but printing everything and providing no means to drop showing default. This made very made it manually while online very hard without external version control to even know what is current status and what had been changes.
BTW, this is still an issue with some manufacturers haven't understood and makes their devices much worse than they understand. Fortigate firewalls CLI is still like this. You can select a subsection of config to print, but it will no indication what differs from default configs. And the whole firewall config is hundreds of kilobytes. Good luck finding what's been changed unless you had saved default config when you started using that device!
When you have thousands to even tens of thousands of network devices in use as many were working you need to be rely that even when you encounter device failure and you don’t have same model any more available, you are able to adapt and find working fix configuring something else doing what is needed.
Binary and any non textual configurations may fit the purpose SOHO category devices, but when network grows even bit larger and configurations more complex there isn’t anything better than text base configs as far as us humans will need to configure and maintain those devices.
Cisco have had issues with software quality for sure, but it was Cisco IOS that was the reason it became so popular and we wanted to use it.
Juniper when it came end of 90's did understand this and made even better configuration system for Junos than IOS was and still is. Cisco have improved things with IOS-XR, but it's still not up to what Junos is currently.
"even better" is an understatement ! I was working on a dual-vendor network at the time, we ran Cisco and Juniper in parallel.
Working on Cisco always just felt like going back to the 18th century, I mean the whole candidate and active config thing alone in Junos was insanely valuable:
Meanwhile, back in 18th century iOS land, a fat finger on the ACL would leave you with a bad day in the office until you rebooted the box via OOB.My “favourite” was “admin”, which was used as the login user, was also short form for “administer-all-port-shutdown” (or words to that effect; it was a long time ago). So if you were distracted when logging in, you could easily soft-lock the entire device meaning you’d have to then run into the DC, find the hardware, and physically power cycle it.
Absolute madness.
However there are also things Junos vs Cisco was that Cisco had made bit easier. Just to mention few
1) a simple and quick way to add aliases to config file, which made life easier with long commands that could be otherwise always in hurry hard to remember. Now later versions Junos they have made this better, before it involved twiddling with XML files OS-level etc.
2) IOS TCL scripting, that proved to be quite useful automation tool.
3) OS upgrades of course with a single flash blob made it easier than upgrading or downgrading FreeBSD based Junos.
The last large scale networking project I did was more than 10 years ago and we went with HP.
Personally I don’t want direct console access. I want a modern DevOps pipeline with version controlled config. I’ve got too many battle scares from accidental typos in IOS. I have no intention or reliving that error prone way of network management again.
Every new product of theirs I’ve touched in the last 20 years has been trash, and even the R&S golden goose is being dragged in that direction with customer-hostile functionality like Smart Licensing.
https://www.cisco.com/c/en/us/support/docs/field-notices/743...
"Cisco Access Points That Are Running, Or Have Run, Releases 17.12.4, 17.12.5, 17.12.6, or 17.12.6a May Exhaust Flash Space, Preventing Upgrades"
Recovery path basically involves either factory resetting every single AP or up to, if it's already bricked, Cisco TAC + you having to console into every single bricked AP you have to pass a secret challenge/response at the bootloader to flash a recovery image :)
Being a mess isn't a problem when you have more than 6 items context.
Otherwise he's just going to replicate all the legacy bugs and questionable design decisions.
Right? ...
[1] https://github.com/enterprisequalitycoding/fizzbuzzenterpris...
Anyone who has used the built in date picker knows why everyone rolls their own.
It's a bunch of text files in a repo with massive churn (all new version 5!), but somehow even the technical debt 'skill' (i.e. text file) has technical debt itself:
https://github.com/DietrichGebert/ponytail/blob/main/skills/...
> ...every ponytail shortcut is marked with a shortcut: comment ... (older code may say ponytail:).
It doesn’t matter if it’s satire. Developers seem to be so vulnerable to all these AI promises of ‘automating’ everything.
BTW it’s terrible. It gave two very hollow suggestions that did not solve the real problem. Hour wasted, angry colleagues, no code change.
What have been achieved using this port?
And how did you make sure the port is correct?
I'd ask "did you make sure the port is correct?"
Followup question: did anyone read the automated test suite?
Silence.
The software rewrite is shelved.
The goal and objective is to deliver value for your user/customers; to the speed, reliability, and assurance requirements your market segment expects.
I suspect that the average Cisco customer expects a bit more stability than $RANDOM_SAAS.
If you have good test coverage, and the tests are reliable, then what's the problem?
If your engineering culture is bad already and you don't have good test coverage or they're flaky or whatever then that is different. If that is the case, it is probably from smart-ass attitudes like yours asking these "clever" rhetorical questions to derail people trying to improve things that led it to be like that. Do everyone a favour and keep your mouth shut and let people fix things.
Why not just ask the coding assistant?
I’ll take the system written in a language that people can maintain and fix
So it's not surprising if they embrace AI.
There's also that Indian cheaper outsourcing companies risk being displaced by AI, if they don't adopt it extensively to further reduce their pricing.
https://www.youtube.com/watch?v=hBA0AH-LSbo
27 is still quite young though and very possibly over confident.
it's the testing in real world usage to shake out all the edge cases and bugs that's hard and that you can't just write unit tests for
Not clear to me what he's doing. Re-implement switching? Simplify features? Reduce LoC? Add test coverage? Either way, AI in that world will be helpful. I'm sure it'll spot all kinds of problems in opaque drivers that are hard to follow from MF specs.
rewrite can be done to quickly refactor and/or close the feature parity, improve the UI/UX for end customer, add missing features or integration needs
I’d love to know what internal services and tools that are included in this, as well as any deliverables that shipped as software products to customers.
If it’s legacy code for IOS firmware, however, I’d be very worried.
-Works according to a defined specification
-Within the bounds of that specification, has a lot of code written to optimize speed, cost, latency, etc - all measureable goals
-Written in c, with all it's weakness
-Given limitless engineering work, some could be ported to better hardware
-the new system could be automatically compared to the old system for functional correctness
All those factors fit well with agent based development.
Btw, many of them are true in general to embedded systems.
Then consider that not all vendors implement the standards identically, because the standards can be a bit woolly and in some cases followed rather than lead live implementations.
There will be a bunch of code that looks useless, but is actually there because of specific interoperability scenarios.
I wonder whether AI is any good at looking at that code,running the system, and documenting those edge cases?
Lots of this stuff should work in theory, you only find out that it doesn't in practice when you try it on real hardware. And there are thousands of bits of hardware.
This whole thing sounds like a deluded vanity project birthed in an ivory tower.
But yes, fact it uses a bunch of very well documented standards with multiple implementations gonna help massively
I think it is a reasonable risk to encounter these bugs again in order to get a more clean and documented codebase. Especially when asking AI to plan and A/B release is also trivial.
Fortune's source was the CEO himself:
> “I’ve given him unlimited tokens because he’s going to refactor code,” Patel told Fortune at a small roundtable dinner. “So I’m like, you can go, you can go nuts.”
I think that Fortune story is better than the Tom's Hardware one. It ties the Cisco anecdote into a larger trend of some people deliberately hiring younger, less experienced developers for their AI projects - because they are more likely to be "AI native".
I frequently find myself in conversations with people who are resistant to AI and turn out to have a very skewed idea of what it can and cannot do.
Or they were reading your comments and articles and couldn't reproduce any of the fantastic results that you were claiming, even back then. Things have indeed changed since the first Opus release about a year ago and LLMs are good at programming now. But still, experienced professionals are not so keen to sign off 1000s of lines of code that they never read, take responsibility for it etc. Also, it is a different job and it will take some time to adjust. Can you blame them?
Moreover, other trades have their own guilds or are otherwise protected through regulation. Can you blame programmers for fighting to keep doing things as they were doing it until recently?
What did you have trouble replicating?
> Also, it is a different job and it will take some time to adjust. Can you blame them?
Not in the slightest. This stuff is really hard. The changes it makes to our production are disruptive and, in many cases, unwelcome.
The main takeaway though is we had one person, in a single month, do the equivalent of at least a couple years of work for a small team. We view bugs and performance issues as a currently inevitable part of the process and we are dealing with them with models just like we do with humans on every other project already. Overall though it is representing an absolutely massive speedup. If you work in an environment where the models make egregiously more mistakes than your human counterparts then I am quite jealous.
yeah, i think they'll survive whatever the outcome
I'm worried for us users of the internet rather than them.