15+ years of expectations around how a website can be run were upended by scrapers for LLM training. They ignore robots.txt rules and seem to scrape as quickly and as broadly as possible. Unlike search engines, it…
Are any sites you'd expect to be on paid-for Cloudflare also blocked? I put https://www.gbif.org/ behind Cloudflare on the business plan, and so far (it's been roughly a year) we've only had a couple of reports of…
Some of the bird data used is available in https://www.GBIF.org (website + APIs + data dumps) but not all of it. For a visualization like this, the place to start is probably the most recent monthly data snapshot which…
With the GROUPING SETS or CUBE features in DuckDB, I think. https://duckdb.org/docs/current/sql/query_syntax/grouping_se...
GBIF track the use of data we provide to scientists, where they later publish papers citing that data [1]. For iNaturalist, the list of known citations is at [2]. In most cases a download of data from GBIF will include…
Global Biodiversity Information Facility (GBIF) | Software developer | Copenhagen, Denmark | ONSITE, VISA Global Biodiversity Information Facility (GBIF) | Senior data engineer | Copenhagen, Denmark | REMOTE GBIF is an…
uk.maven.org went offline years ago, although it's still listed in the mirror metadata at https://repo.maven.apache.org/maven2/.meta/repository-metada... If anyone who can fix that is reading, there's an issue here:…
Global Biodiversity Information Facility (GBIF) | Site Reliability Engineer / DevOps | Copenhagen, Denmark | ONSITE, VISA GBIF is an international organization and open-data research infrastructure, whose purpose is to…
This looks great! I hacked around with OpenMapTiles to generate vector tiles in Arctic, Antarctic and WGS84 projections (the latter with 2×1 tiles at zoom 0) [1]. The time spent waiting for PostGIS to do reprojections…
> [7] https://species.nbnatlas.org/species/NBNSYS0000021551 With data from GBIF you would have almost all of the British records published on the NBN Atlas (about 100 are not shared), plus 2,300 more from other sources.…
AsciiDoctor supports Unicode just fine. We've integrated it with PO4A[0] and CrowdIn.com to support a translation workflow, and so far have one document in Chinese: [1]. Another (tiny) example is [2] with an emoji star,…
I use https://openmaptiles.org/ at work. For normal, Web Mercator maps we just use the free download of the whole world, available at [1]. Running the OpenMapTiles tools to generate a current tileset for a particular…
There are around a million names for plants, but due to changing scientific hypotheses, limitations of science and communication centuries ago, new technologies (DNA sequencing etc) and so on, over 600,000 of those…
The API example[1] gives several sources: USDA.gov, CATMINAT, PlantNet, GBIF, RBG Kew. There might be other sources, but I don't know about all of these in detail, so I'm not sure. Several of those sources have their…
As azeirah writes, subreddits or Facebook groups for people interested in the particular group or location will often help. For general wild plants and animals (and fungi and everything else) there's a computer vision…
Here are some screenshots of RISC OS 3, including the anti-aliased fonts [1]. In 1992, on a 640x480 CRT, fonts rendered on RISC OS were excellent. Losing this when I switched to Windows around 1995 was a big step…
GNU Aspell has an en_GB-ize-w_accents dictionary, which I think is what's needed. The source seems to be here: http://wordlist.aspell.net/ It's the UN's official version of English [1], and my employer's, and it's…
iNaturalist is primarily for wild organisms, but it doesn't matter if they're macroscopic or not. There are some iNaturalist projects specializing in microbes (more than just these two). A project isn't necessary to…
I feel the article would benefit from pictures of wild wombat droppings in their natural state. Fortunately, we have a few on the Vombatus ursinus page [1]. (I work at GBIF.) > Wombats are in trouble. It's not direct…
> nature photos of particular plant and animal species It's not a like-for-like replacement, but GBIF (my employer) indexes a fairly large collection now. You need a scientific (Latin) name for an effective search. HN's…
> The bulk of a 20 million artefact collection would have to be kept in more recently built warehouses That's not often the case. These museums were built to house these collections, and often include rooms full of…
We have records of almost 270,000 preserved animal and plant specimens, for example leaves/flowers dried and pressed or whole insects most likely pinned down into a drawer. This is still useful data, for example knowing…
My apologies, I'm out-of-date with the commercial offerings of OpenMapTiles — I work on an open data project, eligible for OpenMapTiles' free license which has old (currently one year old) tiles. With a commercial…
They're under something like a non-commercial license. Three parts of a map are copyrightable. The map data is © OpenStreetMap under their open license, but OpenMapTiles claim copyright on the cartography of their tiles…
The fastest way, if you don't have resources and don't mind being up to a few months behind, is to use OpenMapTiles vector tiles [1]. The download is an SQLite database with a standard-ish structure (MBTiles), you can…
15+ years of expectations around how a website can be run were upended by scrapers for LLM training. They ignore robots.txt rules and seem to scrape as quickly and as broadly as possible. Unlike search engines, it…
Are any sites you'd expect to be on paid-for Cloudflare also blocked? I put https://www.gbif.org/ behind Cloudflare on the business plan, and so far (it's been roughly a year) we've only had a couple of reports of…
Some of the bird data used is available in https://www.GBIF.org (website + APIs + data dumps) but not all of it. For a visualization like this, the place to start is probably the most recent monthly data snapshot which…
With the GROUPING SETS or CUBE features in DuckDB, I think. https://duckdb.org/docs/current/sql/query_syntax/grouping_se...
GBIF track the use of data we provide to scientists, where they later publish papers citing that data [1]. For iNaturalist, the list of known citations is at [2]. In most cases a download of data from GBIF will include…
Global Biodiversity Information Facility (GBIF) | Software developer | Copenhagen, Denmark | ONSITE, VISA Global Biodiversity Information Facility (GBIF) | Senior data engineer | Copenhagen, Denmark | REMOTE GBIF is an…
uk.maven.org went offline years ago, although it's still listed in the mirror metadata at https://repo.maven.apache.org/maven2/.meta/repository-metada... If anyone who can fix that is reading, there's an issue here:…
Global Biodiversity Information Facility (GBIF) | Site Reliability Engineer / DevOps | Copenhagen, Denmark | ONSITE, VISA GBIF is an international organization and open-data research infrastructure, whose purpose is to…
This looks great! I hacked around with OpenMapTiles to generate vector tiles in Arctic, Antarctic and WGS84 projections (the latter with 2×1 tiles at zoom 0) [1]. The time spent waiting for PostGIS to do reprojections…
> [7] https://species.nbnatlas.org/species/NBNSYS0000021551 With data from GBIF you would have almost all of the British records published on the NBN Atlas (about 100 are not shared), plus 2,300 more from other sources.…
AsciiDoctor supports Unicode just fine. We've integrated it with PO4A[0] and CrowdIn.com to support a translation workflow, and so far have one document in Chinese: [1]. Another (tiny) example is [2] with an emoji star,…
I use https://openmaptiles.org/ at work. For normal, Web Mercator maps we just use the free download of the whole world, available at [1]. Running the OpenMapTiles tools to generate a current tileset for a particular…
There are around a million names for plants, but due to changing scientific hypotheses, limitations of science and communication centuries ago, new technologies (DNA sequencing etc) and so on, over 600,000 of those…
The API example[1] gives several sources: USDA.gov, CATMINAT, PlantNet, GBIF, RBG Kew. There might be other sources, but I don't know about all of these in detail, so I'm not sure. Several of those sources have their…
As azeirah writes, subreddits or Facebook groups for people interested in the particular group or location will often help. For general wild plants and animals (and fungi and everything else) there's a computer vision…
Here are some screenshots of RISC OS 3, including the anti-aliased fonts [1]. In 1992, on a 640x480 CRT, fonts rendered on RISC OS were excellent. Losing this when I switched to Windows around 1995 was a big step…
GNU Aspell has an en_GB-ize-w_accents dictionary, which I think is what's needed. The source seems to be here: http://wordlist.aspell.net/ It's the UN's official version of English [1], and my employer's, and it's…
iNaturalist is primarily for wild organisms, but it doesn't matter if they're macroscopic or not. There are some iNaturalist projects specializing in microbes (more than just these two). A project isn't necessary to…
I feel the article would benefit from pictures of wild wombat droppings in their natural state. Fortunately, we have a few on the Vombatus ursinus page [1]. (I work at GBIF.) > Wombats are in trouble. It's not direct…
> nature photos of particular plant and animal species It's not a like-for-like replacement, but GBIF (my employer) indexes a fairly large collection now. You need a scientific (Latin) name for an effective search. HN's…
> The bulk of a 20 million artefact collection would have to be kept in more recently built warehouses That's not often the case. These museums were built to house these collections, and often include rooms full of…
We have records of almost 270,000 preserved animal and plant specimens, for example leaves/flowers dried and pressed or whole insects most likely pinned down into a drawer. This is still useful data, for example knowing…
My apologies, I'm out-of-date with the commercial offerings of OpenMapTiles — I work on an open data project, eligible for OpenMapTiles' free license which has old (currently one year old) tiles. With a commercial…
They're under something like a non-commercial license. Three parts of a map are copyrightable. The map data is © OpenStreetMap under their open license, but OpenMapTiles claim copyright on the cartography of their tiles…
The fastest way, if you don't have resources and don't mind being up to a few months behind, is to use OpenMapTiles vector tiles [1]. The download is an SQLite database with a standard-ish structure (MBTiles), you can…