The various CERN web pages such as this were a treasure trove of information when I was working on my last novel. I actually included a few paragraphs on Castor thinking of using it as a side-plot, but my editor cut the plot out along with a few other technical niceties. Sigh!
"Castor" was the name of a storage system used for transporting nuclear waste in Germany. There were quite a few protests against shipping nuclear waste through the country.
Wouldn't have been my choice for a software project :-)
I’m a little confused by this submission. CASTOR is the old system that has since been replaced by the CERN Tape Array since ~2020: https://cta.web.cern.ch/cta/
This is mentioned on the page but it’s easy to miss.
- CASTOR at CERN had also its disk centric derivative named DPM (Disk pool manager) that helped to power the LHC computing grid for multiple decades (WLCG) before getting deprecated.
- Interestingly DPM had an architecture quite aligned with the original Google File system even if developed completely separately: (One metadata node, multiple disk node. Design to do Write-once-read-many with very partial POSIX semantics).
- The LHC computing Grid is an association of research centers with their own infrastructure. As such, they had (historically) many diffent storage systems with diffent protocols and interface.
- To unify this madness, an attempt to do a "standard" protocol was made in the 2000s: the SRM protocol (storage resources manager).
In a pure XKCD fashion, it went as bad as you can imagine.
It tried to rely on the tech of the time (XML, SOAP, WSDL) and is a school case of terrible protocol design (bloated, slow, weak consistency, massive server overhead, stupidly complex to implement and quite insecure). The spec are worth a read if you want a good laugh [1].
- After 20y of struggle, SRM was eventually dropped for a more pragmatic and ad hoc solution based on HTTP + xrootd [2]. EOS itself uses xrootd quite extensively. (if this did not change)
- The history of computing at CERN is globally interesting because it is a pretty good image of the evolution of computing and of the "tech fashions" associated with it.
13 comments
[ 0.24 ms ] story [ 19.1 ms ] thread(looks like this submission uses https://castor.web.cern.ch/content/home.html instead of https://castor.web.cern.ch/castor/ the second link does not have the broken image)
Wouldn't have been my choice for a software project :-)
Tape is boring but when an intern / AI / tectonic plate accidently destroys your database setup it is a huge lifesaver
Anybody know what these fancy Oracle tapes are? Is it just their implementation of a regular standard?
This is mentioned on the page but it’s easy to miss.
For the current status of tape storage at CERN see: https://indico.cern.ch/event/1471803/contributions/6967379/a...
For reference, most disk storage for physics data uses an in-house solution called EOS: https://eos-web.web.cern.ch/eos-web/
[1]: https://doc.cat-v.org/plan_9/4th_edition/papers/venti/
- CASTOR at CERN had also its disk centric derivative named DPM (Disk pool manager) that helped to power the LHC computing grid for multiple decades (WLCG) before getting deprecated.
- Interestingly DPM had an architecture quite aligned with the original Google File system even if developed completely separately: (One metadata node, multiple disk node. Design to do Write-once-read-many with very partial POSIX semantics).
- The LHC computing Grid is an association of research centers with their own infrastructure. As such, they had (historically) many diffent storage systems with diffent protocols and interface.
- To unify this madness, an attempt to do a "standard" protocol was made in the 2000s: the SRM protocol (storage resources manager). In a pure XKCD fashion, it went as bad as you can imagine. It tried to rely on the tech of the time (XML, SOAP, WSDL) and is a school case of terrible protocol design (bloated, slow, weak consistency, massive server overhead, stupidly complex to implement and quite insecure). The spec are worth a read if you want a good laugh [1].
- After 20y of struggle, SRM was eventually dropped for a more pragmatic and ad hoc solution based on HTTP + xrootd [2]. EOS itself uses xrootd quite extensively. (if this did not change)
- The history of computing at CERN is globally interesting because it is a pretty good image of the evolution of computing and of the "tech fashions" associated with it.
[1]: https://sdm.lbl.gov/srm-wg/doc/SRM.spec.v2.1.1.html
[2]: https://xrootd.org/
https://gitlab.cern.ch/cta/CTA
Its memory is still alive in CTA, however:
https://gitlab.cern.ch/cta/CTA/-/blob/main/catalogue/TapeSea...