Poll: What database does your company use?
Upvote please if you think it's an interesting question so that more people will respond
Last year I asked this question (http://news.ycombinator.com/item?id=1411937) and I think it was useful to a bunch of people. Figured it's worth asking again and the diffs will be interesting.
377 comments
[ 2.3 ms ] story [ 290 ms ] threadI'm not trying to be a wise guy - it's more of a comment that elementary schools are often last to implement technologically savvy solutions (i.e. solutions that cost a lot of money), so I wouldn't be surprised if you said "yes" to my question.
Graphs are a much more modern and elegant way of storing relational data. I've used Postgres for over 10 years, but it's not a graph database. With graph databases you don't have to mess with tables or joins -- everything is implicitly joined.
And Neo4j is ridiculously sweet -- store 32 billion nodes (http://blog.neo4j.org/2011/03/neo4j-13-abisko-lampa-m04-size...) with 2 million traversals per second (http://www.infoq.com/news/2010/02/neo4j-10), and you can use Gremlin with it (the graph traversal language), which let's you calculate PageRank in 2 lines.
Neo4j is open source, and the Community Edition is now free (https://github.com/neo4j/community).
I recommend pairing it with the TinkerPop stack (http://www.tinkerpop.com/).
There is also a Python open-source Web development framework for graph databases called Bulbflow that is based on Bulbs and Flask.
Both frameworks should be released in the next few weeks.
Actually, storing data as graphs is older than relational approaches. It used to be called "network databases". They were not supplanted for the hell of it, relational databases have certain advantages.
For select applications, object databases are absolutely the way to go. But for most purposes relational is hard to beat.
The graph-database model simplifies much of this and makes working with the modern-day social graph so much cleaner.
Graphs allow you to do powerful things like find inferences inside the data in ways that would be hard to do with relational databases.
How would you calculate PageRank using a relational database? As I said, with a graph database and Gremlin, you can do it in 2 lines.
To see the types of things you can do with graphs, check out Marko's short screencast on Gremlin (http://www.youtube.com/watch?v=5wpTtEBK4-E).
And also check out Peter Neubauer's introduction to graph databases and how they compare to RDBMS' and where they stand in the NOSQL-movement (http://www.infoq.com/articles/graph-nosql-neo4j).
The mismatch comes about for a number of reasons:
> How would you calculate PageRank using a relational database? As I said, with a graph database and Gremlin, you can do it in 2 lines.For something on PageRank's scale, a custom datastore based on matrices and their multiplication makes business sense. Or MapReduce over a distributed key-value store (note that these are both OLAP approaches).
Still. SQL's a bit verbose, but these days we have recursive queries. For Oracle users, I'm talking about CONNECT BY. If I find myself running the social graph every minute, I develop an ETL package that periodically moves data from my write-bound system to my read-bound system with a more query-friendly schema. Depending on how you look on it, relational systems invented "eventual consistency".
More to the point, boring old database greybeards have learnt that OLTP and OLAP are very different use cases. The 3/4/5NF of the OLTP database will be very different from the star schema of the OLAP database.
There's really not much about NoSQL that hasn't already been done, under a different name, by the relational crowd.
I still think there is a place for NoSQL. It's just not as universal a replacement for relational systems as many make it out to be, whether we're talking about document stores, distributed key-value stores, graph stores and so on.
Putting relational databases into their own group is silly. I believe this came about because relational databases have dominated for so long, and this has caused developers to try and fit every problem into the relational database model.
In the early days of the Web, choosing a database meant choosing between relational database management systems -- Oracle was king but expensive, Microsoft SQL Server if you were on a Microsoft stack, and PostgreSQL or MySQL were the primary open-source options (with one being a real RDBMS while the other was basically just an SQL interface to the file system).
Ten years ago the RDBMS was the only good option so that's what everyone used even though many problems and programming languages didn't match up well with it -- it was like trying to fit a square peg in a round hole.
You don't have to do that anymore so relational databases can stop being the one-size-fits-all solution, and instead we can move toward using them for the specialized cases where they're the right fit.
This is somewhat different than a general graph database (http://en.wikipedia.org/wiki/Graph_database) where nodes are not restricted to being a hierarchy.
Also, we use S3 as database for some of our larger datasets where latency isn't a concern and are looking at either Mongo or HBase for large datasets that need faster performance.
All sorts of odd little things, SQL Server.
Various legacy data processing and newer data warehousing jobs, SAS.
New projects, in theory Oracle but there seems to be a degree of resistance. It'd be interesting to see how that pans out but I won't be around there much longer :)
There's nothing quite like sending a DB as an email attachment.
Also, it's awesome.
http://berrange.com/posts/2011/06/22/firefox-form-data-histo...
https://docs.djangoproject.com/en/dev/ref/django-admin/?from...
It's like public storage where you have to sift through everyone else's crap to get to yours, every item is stored in a bulk cargo box and only one customer gets to store their stuff at a time.
If you're interested in db internals, here's a few algorithms that MySQL uses. Note this doesn't cover InnoDB which performs far better under high concurrency loads and offers row level locking, clustered indexes, an excellent caching algorithm, foreign key constraints with cascade, etc..etc...
http://forge.mysql.com/wiki/MySQL_Internals_Algorithms
Something like
It's a quick and dirty caching method that has good persistence, of course, is relatively performant under low loads and easy to understand.Thanks, I'll check out the MySQL thing, but I'm not actually intending to build my own database.
Sarcasm doesn't aid in making points online because people who don't know anything about the issue at hand make sarcastic 'points' as easily as an expert.
So I meant that instead of using SQLLite on mobile devices, pc based local fat clients or in the browser... I now use JSON now whereas I might have used SQLLite before [ I also use JSON where I might have used XML or windows config files etc ]
I did not intend to say JSON is replacing MySQL / PostgreSQL.
But yeah, JSON is great for small amounts of data.
This means I can reuse the same REST style data 'provider' from within a Javascript web UI and within an iOS or Android client app.
So theres quite a few reasons why using JSON simplifies things for me - particularly I notice that there is a whole layer or ORM style boilerplate code that I dont need now [ whether its SQLLite <-> Object or XML <--> Object ].
I actually like the SQL Lite implementation!
"The more of an IT flavor the job descriptions had, the less dangerous was the company. The safest kind were the ones that wanted Oracle experience. You never had to worry about those."
I would say all these databases are worthwhile as long as the DBA is competent. Given that, unless you have a reason to choose Oracle (e.g. familiarity, integration with other applications, a specific Oracle only feature, contractual obligation), I see no reason to choose Oracle. Many companies choose Oracle because Oracle built a solid database/brand early on while catering to big businesses. MySQL and Postgres made great progress in the past decade make them valid alternatives.
We use Oracle, MS SQL Server, and Sybase because that's what the vendorware we use requires. We do have some open-source software that uses MySQL for administrative purposes.
Above all, are you helping a customer? solve his problems instead of compounding it - that my friend is more important than choosing Oracle or Mongodb.
But if your company chooses Oracle, it means that you're bogged down by legacy, stupid company processes and/or clueless managers.
And in most such environments, the harder you try to change it for the better, the harder it fights back, putting yourself in an awkward position in which you are considered the bad apple of the team. So you end up either adapting (not giving a shit), quitting or finding some small project with no perspective for the company (i.e. less controlled) that can bring you pleasure.
Of course, some companies, like Adobe for example, use Oracle when it doesn't impact their core competencies, as it's a safe choice for corporate types. But a company like Adobe doesn't earn money from projects that are relying on Oracle and other projects inside Adobe are also using HBase and MySql and their own distributed file-system that can be queried and so forth.
Either way that quote is correct. I'm not promoting the latest fads (personally I'm not into NoSql unless it makes absolute sense), but you can safely ignore companies that make decisions based on brochures and lap dances.
I don't understand how using Oracle for your RDBMS is suddenly equated with all the things you mention.
http://www.google.com/intl/en/jobs/uslocations/mountain-view...
[If 12 dimensional hierarchical databases with complex logic are your thing then Hyperion is really rather cool in a perverse kind of way].
BTW If anyone here is interested in this kind of space I have had some interesting experiences in building extensions to HFM...
My point, if all you ever need from a database is solved with key/value store then Oracle is not mildly interesting.
Forget price for a moment, any feelings you might have for Larry Ellison, the good/evil nature of the company, or whatever. Consider the software itself.
Many people aren't all that familiar with the basic things a database is supposed to do. The list is large. I can't possibly give it justice off the top of my head, but here are a few that come to mind: make it possible to see data in a consistent state (as of a single point in time), without being blocked, even as other people are changing it; support "transactions" -- a group of requests that either all succeed or all fail -- so that the database cannot end up in an inconsistent state; guarantee recoverability to a consistent state even when people change their minds, statements fails, power fails, hardware fails, or all of these things happen at the same time; and many, many other things like security, support for the relational model, SQL, joins, etc. Real databases aren’t simple key-value stores.
Historically, Oracle obtained a huge lead in market share because it delivered the most complete mix of these basic things long before anyone else.
Something not widely recognized, but that should be obvious to programmers, is that low-level architectural decisions have a huge impact on how well a database performs these basic duties. Oracle maintained its lead for a long time because it got many of the low-level architectural decisions right. It is really hard to catch Oracle if you are trying to polish up a bad locking model, for example. Oracle still does the basics better than most. That, in and of itself, is a reason to consider Oracle.
Oracle remains viable, even as others continue to catch up, because Oracle builds on its solid foundation by adding additional capability and features relentlessly. I challenge you to read the new features guide for any new release of Oracle and to remember just the names of 20% of the new features. The Oracle documentation, as of 6 or 8 years ago (10g), was 40,000 pages. No telling what it is now. I can tell you this. If you have something you need to do with databases, Oracle probably figured out how to do it a long time ago.
I, personally, am a huge fan of PostgreSQL (and it’s freeness), but I recognize that Postgres is never going to be able to touch Oracle in features. It’s impossible. Working with Postgres is just going to require a lot more manual labor. Some things aren’t going to be possible. Performance may just have to suffer sometimes. Let’s hope Postgres does the basics well (it does). That’s the most important thing. But when it comes to building spacial indexes on hierarchical dimensions, or whatever, Postgress just isn’t going to have a feature for that. I’ll have to figure that one out for myself.
One key point: if you need to build a large, high-performance, data-driven application that provides nearly instantaneous response for thousands of simultaneous users, Oracle is one option that can get the job done. If you need to build something huge, say billions of rows, that provides nearly instantaneous response to dozens of users, Oracle is one option that can get that job done. So no wonder that some companies consider Oracle.
The last thing I’ll mention is that it takes a lot of time to learn something like Oracle. I would be surprised if after a year of using it, for example, that you can really make it hum better than any of its top competitors. If you do use it for a while I think you’ll find that it is really good at the normal things and in a different league when it comes to the unusual things. This comes in handy when you are being paid to get things done (by a company that can afford Oracle) -- hence the reason that most startup-oriented people don't have much appreciation for Oracle.
Are these good things?
We can also reverse this argument: if you're using Oracle, you will never have a use for 99% of its features.
If you need Oracle, you'll know it. If you don't know that you need Oracle, you don't need Oracle.
I run MySQL, Oracle SQL Server, hundreds of databases, a couple in the 'many thousands of queries per second' range.
There are reasons for each database platform.
In this case, yes it is, Postgres probably has the best type system and extensibility of all SQL databases. GiST and GIN are very good technologies that also exist in Informix.
If I had to choose something that is missing in Postgres and very far away for sheer want of implementation effort, it's parallel execution of queries.
Oracle RAC is also quite a crazy and neat feature, except when it has problems, in which case you get even more crazy for that crazy.
On the other hand, Oracle is also not a strict superset of the features in PostgreSQL. For example, 9.1 has the only credibly fast implementation of SERIALIZABLE level isolation that I am aware of. There are also interesting features like "index exclusion constraints" that have no equivalent in Oracle, and solve problems hard to solve otherwise (example constraint: there are no overlapping circles in this table)
1. I build products on Oracle at a startup.
2. Oracle has many compelling features that really don't have first-class open source alternatives: OLAP, encryption (wire and at-rest), VPD, materialized views with query rewriting, object and document storage, monitoring and tracing, among others...
3. Oracle got more compelling since being offered--license included!--on Amazon RDS.
There are a ton of responses for Microsoft; why doesn't anyone question that? Actually, I'm curious now. Is there a new incentive for SQL Server? Maybe something in BizSpark that makes it attractive?
Sybase was better than Oracle, and Postgres was just as good as Sybase at that time. But SQL Server (version 2005 on) has stood taller than all of them in my view. Excellent management tools, stable, reliable and no performance issues for our apps. Most of our apps are write light and read heavy with thousands of users hitting the db - served off one Win 2003 server (we have a warm spare).
So there is some truth to your statement about nobody choosing to use it - for us the primary motivation to not use it internally was cost. At our customer premises they picked up the tab. We have MSDN so sql server development licenses are not an issue. Have not met a Microsoft salesperson yet!
I've never had a bad experience with it, it needs a lot less maintenance than Oracle does (or did, at least), it performs pretty well (we're an insurance company with large datasets), is integrated in a .NET stack (while you might not want to go there, especially as a startup, it is very nice for corporate work), it came with Reporting Services which we used to replace Crystal Reports, it has lots of (admittedly non-standard) extremely useful SQL functions, data types, etc. And I love SQL Server Management Studio and the other Microsoft tools.
MySQL felt like a toy database in comparison (especially the management aspect), and the non-relational stuff is out of the question for now. We might not have done due diligence by not looking at other alternatives (notably Postgre I guess) but they seem like a poor fit given our developer's strong Microsoft-centric backgrounds. Most of us also have at least one Microsoft training course in their SQL server, and they have very strong support in my country (Uruguay) against nonexistant for most other platforms.
Also, go Postgres! Woot!
Fortune 500's will pay you to spit out graphs, but they don't use bleeding edge tools like Redis.
Anyone who uses Redis in any serious fashion already knows that the numbers you show on the homepage are easy to get to. Just write a script to pump them into $graphing_software.
My advice would be to give the current product away for free (yes right now), making the barrier to entry lower than doing it yourself and capturing future customers. Then focus on selling people things that are hard, like real actionable intelligence. Everything you charge for should answer a question like "what", "who", "where", "why". Free stuff should answer questions like "how many".
[Edit: Just realized you are the Hoptoad guys (of which I am a happy user), so I know you have the potential]
Also, while I love Tokyo's speed, it is just a better bdb - which is an impressive feat, but I can see why a lot of people would use something slower but more featureful.
Next question, there's karma here?
I'll leave it to you whether you call it a database :-)
Access plays well with everyone (including Oracle, the MS Office family, and Sharepoint), can be heavily extended if you know VBA and are willing to hack a bit, and can often be handed off to a non-technical person for support. The trick is to try to use the same coding/design standards that you would use in a real langauge - the internal parts of my Access/VBA work is influenced by Python and C++.
And from a consulting perspective, it has the advantage of being aligned with what 90% of your Fortune 500 end-users are familar with (MS Office)... simpler handoffs.
That being said...not my first choice for after hours work (do a lot of MySQL, looking to step up to Postgres).
Incidently (for the other thread) - one really nice thing about oracle is their statistical analysis functions - very nice set of tools, co-located with your data, and can be accessed through SQL. Only free solution I've seen that is competitive in terms of functionality is Postgres...
Just to name a few:
* SQL Server
* SQL Server Express
* SQL Compact 4
* SQL Azure
... and one of the 2010 poll: http://koldfront.dk/misc/hn/database2010/data.png
(Updated every ~4 hours currently; I will be fading out the update rate.)
The two sets of data ought to be shown in the same graph, really.
Edit: I have now joined the two years in one graph, and plot the percentages to make it easy to see the change: http://koldfront.dk/misc/hn/database/data.png
On a lighter note, does this support the theory of HNers who complain that HN is not the same any more and it has been generalized?