I'm years out of date, but it competed well with ArcGIS until the expert level, to the point that I gave up my ArcGIS license where I was working and didn't miss much of anything day to day.
QGIS also has server and headless options, plus lots of the QGIS core is importable into python scripts.
I was excited when I learned about a way to create custom georeferenced map PDFs (say, based on an official, but not georeferenced, national park map document). It used qgis, plus one or two other open source tools. It seemed straightforward, if a bit tedious. But I was never able to get the resulting files to work properly on my Android device. The instructions said to use an app called avenza maps, which would understand the georeference data, and show your maps, other maps, plus routes and tracks, all on one display. It just didn't work consistently for me, sadly.
From what I understood, qgis was doing its job properly. It was a neat tool. I guess the whole stack for this use case was just too fragile.
QGIS is the swiss army knife of geospatial computing.
I use it daily, and frequently, for all sorts of work building a location API SaaS company. It's a sterling success of open source, on par with as impactful as Linux, for the geospatial world (IMO).
I tried to perform what I thought were basic tasks of drawing a few line segments and measuring geodesic distances between two points, but I needed a janky plugin for the former and never figured out how to do the latter.
I really do want to learn it however, since it's a lot easier sometimes to work interactively than doing everything in Python code, and I know QGIS is supposed to be a very powerful tool in general.
I learned QGIS arguably the hard way (creating a new non-Earth global basemap from scratch, with no prior GIS experience) but:
- The QGIS Training Manual is good. Not great; don't be afraid to go out of order after the first few modules to get to parts useful to you, because the module organization after "Creating Vector Data" stops being terribly linear. https://docs.qgis.org/3.22/en/docs/training_manual/index.htm...
- https://www.qgistutorials.com/en/ is more project- and task-oriented, and where I learned georeferencing and digitizing (drawing over raster maps)
You can measure distances natively in QGIS, the symbol looks like a ruler. Adding permanent points requires adding a layer, e.g. a shapefile of points.
That being said, as I've outlined in the article, I certainly haven't properly learned it...
But having dabbled in Python/Folium and the like (most recently did geo vis with Superset for another article), some of the datasets and functionalities were familiar (I also had to do point-in-polygon problems in Python, way back when, so maybe that mental scar helped :) ). I found random blog articles and stackexchange posts for specific questions I got stuck on the most helpful.
For instance, I have this link saved: https://gis.stackexchange.com/questions/421467/not-geotagged... , which got me down the rabbit hole of "Oh, you can make a style a function of other things? You can read EXIF? How neat!" (where I then found some other posts).
What I will say, almost everything online is out of date - the GUI changed frequently, plugins are now native functionalities etc. The HTML tooltip thing I did is also a modification of an older blog article (which I'm afraid I didn't save), whereas the contour line styling I took verbatim from the article I linked in that paragraph.
What search terms would one use for finding roles in companies like this, as an experienced dev with a strong background in GIS and open source tools? E.g. Postgis/Pydata/mapbox-gl et. al.
GIS is a huge space, and you could specialize in one niche for years or go more general, so it's really based on what you want to do.
I'd say there's three or four general areas: mapping & map data (mostly focused on visualization), geocoding & search (focused on helping people find things in natural language, think text -> coordinates, text -> point of interest -> coordinate), routing (focused on the mechanics of getting things from point a to point b efficiently), and finally a huge, diverse bucket of "other location APIs" (anything from geofencing to timezones to ETL specialization).
I'm happy to chat about this. We aren't hiring, but there are lots of companies that are.
The moment I graduated and lost access to ArcGIS I got out QGIS and become ridiculously empowered. Yea Arc has Python APIs for some things but QGIS, while rough in the UI department, gives you powerful access to everything. And it plays so well with other things: check out my presentation on using QGIS in robotics (pdf) https://roscon.ros.org/2018/presentations/ROSCon2018_Unleash...
Disclosure: I work for Esri, creator of ArcGIS, so obviously I'm extremely biased. But I'm not in sales, just a lowly dev fwiw.
QGIS is great software, and I respect what they're doing.
ArcGIS Online is a pretty great option and the Living Atlas [1] makes it really easy to consume a lot of data (a bunch of NASA datasets were recently added) and make cool maps. ArcGIS Online ends up being almost like Dropbox for geographic data, but instead of just files you can get full services, and a suite of apps for taking advantage of the services. That includes taking maps offline, making field edits, and syncing them (although that starts to cost money).
You can try the map tool without signing in, and that gives you a good idea of what you can do in the browser (custom popups, charts, tables, filtering, symbology, clustering, cartographic blending, etc). https://www.arcgis.com/apps/mapviewer/index.html?layers=dece...
The personal use subscription is $100/year and includes the full desktop software and cloud storage. [2]
I think you can get very far with ArcGIS Online. There are things you can only do in ArcGIS Pro, but basically everything shown in the article (and a lot more) can be done in the browser. And you get a great set of apps (including Field Maps and QuickCapture) for field editing/data capture.
In my personal experience with using ArcGIS, I have only needed Pro in two situations. The first is when I'm trying to georeference a raster (I just bought one from SkyFi). The second is when I've bought a tutorial book that is based around ArcGIS Pro (e.g. Modern Policing Using ArcGIS Pro).
I won't say more, but I have personally been impressed by the rate of improvement in feature capability in ArcGIS Online. I am not aware of any plans to move Pro to macOS, I feel your pain there.
ArcGIS Pro is just the name for the current iteration of a desktop suite (after ArcMap). ArcGIS Online lacks many of the „classic“ GIS features, especially raster features.
I think it’s just that the web map viewer is a new product and that feature hasn’t been implemented yet (but I’m not on that team and don’t know their roadmap). And it’s not really a paywall thing since the personal use license gets you both online and desktop apps including Pro.
ArcGIS online is absolutely not an alternative to ArcGIS Pro. Not even close. Nobody spending more than a small amount of time using GIS beyond being an end user is going to find online a worthy replacement for Pro. It's a publishing platform with a few analytical tools that each cost credits to run, not a spatial sciences package. Online is good for data capture and integration, but not for someone who wants to perform regular or more complex analytics.
There are currently no plans to rewrite ArcgGIS pro for M1 mac hardware according to ESRI. The move to Pro should have made it much easier to do, but its still a huge task.
The ESRI community have recently had a lot of success with M1/2 hardware, Parallels 17+, Windows 11 arm, and ArcGIS Pro. Just note that you won't be making 3d videos with it comfortably. Everything else seems to be working very well.
I installed ArcGIS "home edition" in Windows 10 for Arm on an M1 via Parallels and it performs rather well. QGis has never really been my thing, I guess I learned Arc (3.2) at too young an age and this old dog wont (no time) learn new tricks.
Their desktop version doesn't. Their target audience is large to very large enterprise. Their customer base is overwhelmingly windows based. They know this, so they've shrunk their dev costs to account for it.
ArcGIS is painfully slow. To apply symbology, you need to go to toolbox, select apply symbology, select the layers, select the symbology file, then make sure to select maintain ranges which for an absolute no sense reason is not the default, then click on apply.
ArcGIS then proceeds to apply symbology one layer at a time, taking between 30 to 75 seconds per layer to apply it.
In QGIS, right click in layer, copy symbology, select other layers right click, paste symbology. Done in 3 seconds.
And like that I could be hours and hours talking about similar experiences.
ArcGIS might be much more powerful, but honestly it isn't worth spending 45 minutes to what in QGIS can be done in 5 minutes.
A lot of people this post are trying to reach don’t know about ArcGIS. As someone who does know about the GIS field, it is frustrating to see post after post about how to do GIS the hard way. The article is fine, but the whole process could have been much easier using the ArcGIS system, and instead of a PDF you’d get full REST services and a suite of apps to work with them.
This post doesn’t mention ArcGIS at all, except for as a data source that your county government probably uses.
I remember when I didn’t know about ArcGIS and thought it was cool to build apps that put points on MapBox maps - which I later found out is a waste of time, because ArcGIS already does that job well.
I think this post leaves people with the impression that GIS is niche, or difficult, or not accessible to lay users. My personal opinion is that recent advancements in Web GIS make the field about as accessible to people as things like word processing, while QGIS is stuck in the ArcMap days, catering exclusively to trained GIS professionals.
It is absolutely meaningful to point out that you don’t need to be a trained data engineer to do this kind of work, and that this 4,000 word journey could have been done in 10 minutes of clicking around user-friendly software.
QGIS and PostGIS were my jam when I worked in that space. We were an ESRI shop so Oracle/SQL Server with SDE (topped with ArcGIS) were the official tools. Some of us were always looking for ways to subvert the culture by building tools based on open source stacks.
One of my favorite experiences from that era: We were meeting with a few ESRI reps for some integration stuff. The lead hot-shot was on his phone playing around during the meeting. He was basically on autopilot. The other two folks were working with GeoJSON response convertors. I said, "I built one of those with TopoJSON". One guy said, "I've never heard of it". I showed them how it was much more efficient and used splines instead of points. The lead dropped his phone and said, "I need you to tell me MORE about that". I showed them the service. They invited me to lunch, I politely declined and said, "today's my last day so I have a ton of things to wrap up". I do miss that realm sometimes.
It's not the software licensing, but the independent consulting entourage that's the problem : internal confidential big four accountancy information told to me without any applicable restrictions on my repeating this put the cost of each Oracle database instance deployed in the F500 at over one hundred million dollars, due to the third party make work.
But it's the other way around. Esri offers so much in the education sector apart from software licenses in a complete package, which you unfortunately have to search for in the FOSS sector.
The business world needs to stop supporting Microsoft Office. The creative world needs to stop supporting Adobe Photoshop. The industrial design world needs to stop supporting Solidworks/AutoCAD. Everyone needs to stop supporting Microsoft Windows/Apple OSX. etc.
I've been telling people for years they should be using GIMP and Linux but nobody cares. At least Oracle's stranglehold seems to be weakening.
Yes, that is the apex of their human capital Ponzi scheme. But, you’d have to wait until the current profs who only know esri retire. Or until they put in the effort to learn how to use foss tools. The only motivation they might have, is a strong belief in foss or pressure from students who know esri is a trap.
That’s not a very good take. Esri is not a Ponzi scheme in any way. There isn’t an alternative for serious GIS. QGIS is a good open source tool but it is nowhere near esri.
As a systems engineer who was hired explicitly for a massive migration from 2 disparate ArcGIS Enterprise environments to an AWS one...
Yeah, there's a massive amount of essential features just missing from OSM and QGIS. Like, even the most basic - layers. They don't even really exist in OSM side of things. Whereas if I'm doing travel-time-bands from medical centers, I absolutely must use multiple boundary layers.
Then again, I'm an engineer, not a data scientist. My colleagues (all data scientists) do all sorts of crazy awesome work. Again, unless you're hacking R/python code and piecemeal it together, you're using ArcGIS. Then again, "ArcGIS Notebook" is a dockerized Jupiter system for doing just that in ArcGIS enterprise.
edit: As for me personally, since I'm working in geoinformatics, I've been teaching myself the hows and whats. Im pretty proficient at both ArcGIS and QGis, as I'm trying to learn them side by side.
If I am doing my own projects, I will use QGis. That's not even a discussion. I'm not about to use proprietary systems to lock up my data.
No, because any further details would highlight where I work and specifically what office.
Lets just say the data intersection of: density and geocoding of users, density and location of office sites, travel time bands, and service level expectations all have a part in the final calculation.
The data is either base layers imported, or calculated layers. There is no one "comprehensive layer" nor does that concept make sense. Different groups handle their own data. We import it in and use their data for our final result.
(In fact, its the OSM approach of everything as 1 layer and tags everywhere is why a DB import and data handling is so obnoxious.)
What features do you think are missing in QGIS which hamper you in your work? I am asking because I am developing a spatial query engine which will work directly with openstreetmap.
I’m not sure that I believe you know what you’re talking about. Not trying to be offensive in that statement, but “can’t back up my very broad, generalized statement with any detail or explanation” comes across as though you’re afraid to reveal your lack of understanding about the details you elided.
You’re saying he doesn’t know what he’s talking about? He is saying you can’t really do GIS without multiple layers. This is basic knowledge for anyone working in GIS.
I am not claiming to be world class expert in GIS and English is not my mother tongue. This comment by tarotuser is confusing me "Yeah, there's a massive amount of essential features just missing from OSM and QGIS. Like, even the most basic - layers. They don't even really exist in OSM side of things. Whereas if I'm doing travel-time-bands from medical centers, I absolutely must use multiple boundary layers." Layers have existed in QGIS for a long time. What are those "massive amount of essential features just missing from OSM and QGIS"?
Explain how qgis doesn’t allow multiple layers. I have a project open right now with 6 layers. That to me seems like “multiple layers.”
And how would that concept even apply to postgis? A postgis query can operate over many tables.
And there are a variety of open source tools that allow multi-dimensional data structures. That’s multiple layers in multiple dimensions.
So I’m hoping, if there’s some aspect of a GIS layer I don’t understand, he can explain it. Qgis totally supports multiple layers in the sense that most people think of it.
He doesn’t have to divulge “secret information” to do this. If he is referring to some aspect of the qgis architecture he can just say so. Qgis is open source, their GitHub repo is clean and well documented. Im merely asking for him to explain why it handling multiple layers is somehow not really handling multiple layers.
What you’re describing sounds like it fits perfectly into PostGiS. You can have tables and you can normalize your geospatial data. With SQL, and temporary tables, you can build absolutely any analytics.
I worked in a GIS Lab with an ESRI endorsed all Arc stuff textbook author/instructor for a number years. Working on similar projects, for anything remotely complicated, I could implement anything he could do, with some python and PostGIS SQL, usually faster, and instantly reproducible.
ArcGIS is a crutch for people who can’t and aren’t willing to program. This goes for all data manipulation tools that aren’t focused on visualization.
If you have an ETL pipeline, probably best to get GIT involved.
They explicitly target the universities as part of their business model. That’s why I say it’s a human capital Ponzi scheme. They give products out to students and educators way below cost, with the specific intention of making the field as a whole dependent on their products. This is well known and pretty explicit. If professors and students had to pay the same costs as private enterprise for esri products the demand for their products would plummet. This is their business strategy and it’s well known and more or less undisputed.
Whether or not they are a good company that provides quality, unparalleled products is not a point I’m contending. They are not competing on that alone. That’s my point.
That author doesn't say Esri is a scourge; more like it's just generally bad to have only 1 provider in a space -- and that it's up to customers to change that by voting with their feet.
I think Esri is (and has been) in a very similar position to Microsoft's in the late 1990s -- having achieved market dominance, they feel like open source software is the biggest threat to their business. But I think the presence of QGIS is creating competition, which is nothing but good for the industry.
ESRI have similar network effect to Microsoft. No individual product is that great but taken together it is very competitive. And ArcGIS online is developing that network effect in a similar way to Office 365.
Also, the GIS space tends to have crossover between different departments in companies. You have a mix of IT departments, GIS specialists outside of IT departments managing infrastructure, data management, casual users etc. Each has a different budget and a different competitive environment. ESRI sits as a compromise between these different groups. Each group may be able to adopt a competitor but not gain traction outside their group.
Not having done much in terms of GIS work, I never had to deal with ESRI until last week.
I was on a call to ask some ESRI rep to add some labeled points to my client's existing map tool. It was a somewhat surreal, weird experience where I got the feeling they were making the work seem much more difficult than it was. Their estimate turned out to insanely off the (my) mark, at eye-watering hourly rates.
At first I thought they had enough business and didn't really care about us. But reading this thread, it seems I was wrong. They are the Oracle of the GIS industry.
In their (possible) defence, stepping outside the the narrow ESRI path of the way you're supposed to do things makes your life a massive pain. I used to work for a company that (among other things) developed tools for clients on top of ArcGIS Online. And I've had many calls where a client asks for a small and very reasonable change, and I embarrassingly have to explain that that is not the 'approved' way doing things in ArcGIS Online, so trying to make that change will essentially require us to rewrite a whole lot of components more or less from scratch and might no even be possible.
I've spent a large number of hours trying to fight ESRI software, lost more fights than I've won and even my victories have mostly been pyrrhic.
Thanks for the perspective, I'm entirely new to the space. We're supposed to use ArcGIS. So if this turns out to be the least painful path, I'll have to learn to deal with it...
ArcGIS the desktop tool, or ArcGIS Online? The desktop tool at least has a pretty complete (but pretty annoying) python API so that if/when you have to, you can circumvent the GUI and automate almost everything. ArcGIS Online is unfortunately extremely inflexible, and things that should be easy are often impossible.
Is there a better option for raw base tilesets? When making my app mapping world tides (https://solunar.pages.dev) I tried OSM for instance but it only had raster tiles at LoDPI resolutions available. ESRI on the other hand had full vector support in a variety of formats^ and provided great docs for integrating with FOSS rendering toolkits. The free tier seems generous, though once it runs out the price increases sharply.
^ For instance, a really neat one that renders ocean features in as much detail as the land typically gets
Mapbox's free tier is an order of magnitude less permissive. They even charge for loading their JavaScript: $5 per kiloload! Feels like the best approach if I'm not trying to commercialize is just to encourage people to self-host using their own API key.
Assuming the service ever even gets to 2M tile requests/month. That alone would be quite the accomplishment.
Though with Gaia going to hell, something new needs to come along.
The cost and effort of deploying something like that in production is decidedly non-trivial. I know because I've done it. Unless you need the customization and flexibility such a solution offers, it's rarely worth it.
Wow, these are gorgeous! Can I ask where you get your DEM data and at what resolution? I’ve been wanting to play around with some relief maps of various bioregions (eg the Great Lakes watershed, Cascadia, etc), but I’ve had trouble figuring out where to find data at the right resolution
Where do we get it?
Only publicly available sources. Usgs has a great portal. Private data is too expensive to get. I was quoted 6 figures for a larger area. They were going to fly a plane and capture it :)
What resolution?
Totally depends on the area the customer would like to cover. If it’s their ranch or property, we usually need 1-meter. If it’s a mountain range than 30-Meter works.
It mainly depends on the resolution limit for 3D printing. So it also depends on the size of the model they want.
Unfortunately not all areas are covered with high res
For the benefit of those who aren't familiar: SRTM was the space shuttle radar tomography mission. 30m resolution is now available (I remember 30m being released for the first time in the 2000s sometime I think).
What are you using to get it into an STL? I've had OK results with DEMto3D plugin, but it has some weird artifacts that I can't seem to get rid of. Are there any better options you're aware of?
I actually use DEMto3D. It's touchy, but I do post-work on the .stl/3d model in blender so it works out ok for me.
If you have weird artifacts, I'm guessing that is due to the underlying data vs QGIS itself. Have you looked at their documentation (https://demto3d.com/en/)?
use these plates on a larger scale to make terra-cotta impressions and turn them into 'chia-pets' such that you can grow micro greens on the landscape of a particular area.
You have fewer products for sale than I was expecting.
I live in NE Los Angeles, which has the Verdugo and San Gabriel mtns, plus Mt Washington and other modest peaks - I think it would look great in this style. Especially because all the development would be excluded.
+1 to this, I can think of a few places id order for but i couldnt figure out how. Did i miss the link? Looks like it refers to "custom orders" but i wasnt sure how that works.
For every new location you do the process looks like:
1. Get the data and prep it for print (fixed)
2. 3D print it (fixed)
3. Rubber Mold (fixed)
4. Wax Model (variable)
5. Bronze (variable)
Steps 1-3 are 40-60% of the costs. So I haven't put the money out of pocket yet to put up new locations. I've let customer's ask first and then do them.
This piece of software brings back fond memories. It was just amazing to discover this after struggling for days with commercial ill fit tools and suddenly feeling i could do anything imaginable. We used this to model and visualize malaria cases and population distribution and plan health care monitoring in liberia. I think the only thing comparable to this feeling is discovering VLC to play video files as a teenager.
I'm trying to find a good project to work on to start learning QGIS, and what you described sounds interesting. Would you happen to have a write-up of this?
sorry not really, but i recommend something where you have an interesting question and at least one dataset you need. A nice starting point is always to get some of your private data exported google and others have about your location history and come up with nice questions. another approach is to browse the plenty of public datasets and use gapminder for inspiration about topics or questions you want to answer.
QGIS is absolutely one of the biggest open source success stories. Its interesting to see the funding and delivery of new features is largely done in Europe where in the US it is extrmely hard to not use ESRI products.
A couple of packages used by qgis worth calling out - postgis which really grew up with qgis, and pdal which is a relatively new but incredible package for working with point cloud data.
Yes, Europe is going the OSS path. I work at University and almost everyone I know in GIS fields switched to QGIS in the past 5 years. Most government departments are also (trying) to switch, some succeeded already (others are waiting for important people to enter pension, so their replacements are more OSS minded).
QGIS is very powerful, but it's not exactly user friendly. The UI is hard to learn, and important functionality is buried several layers deep in non-intuitive drop-down menus and buttons.
I think a simplified GIS program with stripped-down functionality and an intuitive UI could be a big hit. Think of SketchUp versus SolidWorks or ProE.
No argument there.
I've worked with ArcGIS, Autocad, Sketchup, Civil3D, and QGIS. Though clumsy, QGIS is the most powerful by far of these applications.
In many ways that is what we're building at Felt (where I work). We don't offer much in the way of analysis, but we make it very easy (IMO of course) to share your maps.
I think part of the question (and problem) is, which part of QGIS do you focus on? The digitizing and geometry parts? The map-rendering part? Georeferencing? Vector and raster analysis?
Or to put it another way, what's the workflow that a simpler, more opinionated interface would solve, and what features could or would you sacrifice for it?
The most obvious ones. I'm still very new to GIS, but I find it amazing how difficult it is to simply draw some rectangles (of some specified dimensions) and lines on a map, and cleanly label the edges with distances. Annotating, moving, rotating things... is quite a pain compared to modern UIs. So I wonder if there's a slim and beginner-friendly open source version of QGIS.
"Most obvious" is still pretty subjective, though. It depends on how you wind up looking for a GIS tool.
- A lot of people come to it wanting to edit or annotate existing city/street maps with geometry, like you're describing.
- My first encounters with it were trying to plot points on a map to build a heat map, in a data analysis context, and I was coming from Google Earth/KML.
- The next time, I wanted to georeference raster maps to make more accurate vector data, and I was coming from Inkscape and Illustrator.
- Another popular use case is to work with topographical and height data, which is another workflow with mostly another interface in QGIS.
- And all of these can have very different output goals - slippy maps with raster or vector tiles, topo maps, 3D renderings and prints, GeoJSON and PostGIS data.
The "most obvious" app for each of those use cases would each be different, with different interfaces. And I can see where that would be good for beginners, or for limited use cases, but at some point each of those use cases intersects with another. Drawing bike lanes on a map quickly means working with topo data. Georeferencing a new base map eventually hits plotting POIs and drawing things like political borders as geometry. Outputting a heat map from data you plot in QGIS can work, but then you need to plug it into a database at some point so the data can be updated in a shared or automated context.
IMO what would be more useful for QGIS is a ground-up, no-assumptions review of its UI similar to what Blender did in 2.8/3.0. Find where they can consolidate redundant or conflicting tools and interfaces, redesign their mouse/keyboard interactions and tool palettes around how users intuit they should work, and provide alternate modes for the more complex workflows that existing users are accustomed to. Blender's redesign proved pretty well that an esoteric UI can be streamlined and modernized without sacrificing power user workflows.
But I don't know if the QGIS UI community is as large, organized, or focused on modern UI design as Blender's, which was a rather unique case in FLOSS. And it'd be difficult to divorce QGIS's GIS data-editing engine from its interface as a whole to build standalone tools with smaller UIs focused on tasks instead of toolboxes.
Plugins might be able to do this, which could be an interesting approach. Instead of adding features like plugins tend to do, one could ship a "flavor" of QGIS with pre-installed and -configured plugins that provide narrower, task-focused interfaces around existing features.
Some years ago QGIS was useful to me in spite of its UI: I got a PDF map of certain infrastructure that was not georeferenced, and I was able to correct its projection to Web Mercator with great acccuracy. But at the end I was somehow unable to find out how to export the data! QGIS did show me with the transformation matrix it had built and I wrote a program that applied that matrix to my files.
In the vast majority of cases you cannot turn a PDF into spatial data, except for georeferencing it as an image. There are some exceptions, such as when the PDF already includes georeferenced vector data, but its very unlikely you had that sort of file given the issue you described.
You likely either:
- needed to export an image of the data (raster);
- needed to digitise the data within;
- needed to extract the vector data first from the pdf and work out how to deal with it.
Opening a pdf without appropriate data structures into a gis software package is akin to taking a picture of a billboard that has a printed image of a map on it and then expecting to be able to do anything with that picture. The pdf is a bit better, but not by much unless it began life as a georeferenced pdf with vector data maintained.
The PDF was a map of infrastructure that was likely produced with ArcGIS.
By the time I got it, it was just a vector image.
The infrastructure in question was pipelines, represented in the PDF by lines, so georeferencing them was enough to get spatial data adequate for the application.
I used to program python plugins for QGIS, I really miss it. It made me feel really useful, I'd write visualization algorithms one day and the next it was deployed to the whole company and people would email me telling me how much easier it made their job. I don't think I've ever felt this valued since, every other job I had was a small part of a big whole.
Hello, would love to chat with you about QGIS plugins. Email me at devjobs@anor.io if you would be interested in making a QGIS plugin for the solar PV industry :)
Ideally a conformal mapping projection with minimum scale error.
I'm saddened that complex-number polynomial expansion fell out of vogue and fear that was the last wall that kept the muggles out of hard core map projections.
I love exploring open GIS datasets with QGIS. It's incredible what this tool can do and I keep discovering new features or plugins all the time, which keeps it interesting. The amount of publicly available GIS data combined with tools like QGIS and PostGIS can be a big time saver. I don't work in the space, but occasionally run into problems requiring some form of support for geospatial data. Need a list of US cities and postal codes along with their respective coordinates? Easy. Find the dataset, import into QGIS, export to PostGIS. Done.
I spent a decade in the GIS space, ending back around the final days of ArcGIS 9.x, or very early days of 10.x. I also remember various alternative and early open source efforts of the time.
Then, about a year ago, after several years away from the scene, I got some mojo back and started investigating modern QGIS. I was very pleasantly surprised, it immediately felt like an upgrade from what I used to use. And also felt like coming home, in that it was obvious and intuitive how to accomplish the standard functions. If someone had put me in (that famous thing) a blindfolded time machine for a zero-year trip, popped me out, put me in front of QGIS and said "Welcome! This is ArcGIS 9.6!", it wouldn't have been at all unbelievable. If you know what I mean.
No it's not perfect. Casually and hastily wing your way around a project, copypasting here and there, setting up all kinds of parameters and advanced display bits&bobs, and - like most complex geospatial software - you'll, once in awhile, find yourself unceremoniously dumped out onto the desktop with a software error. So a highly robust and restore-able workflow is still essential. But regardless it's up there with the very best packages I've ever used.
Interesting, I have had the opposite experience to that. I spend a lot of time in ArcGIS late 9.x. and early 10.x and QGIS is not even remotely intuitive to me. You can drop me in ArcGIS Desktop or Pro years later and I can still intuit how to do everything, but I have to Google how to do anything in QGIS.
If you can, get yourself in front of an up to date version of ArcGIS Pro for a little while. The user interface will feel entirely different, and some people don't like it because it's different, but the entire package feels significantly better.
Importantly, a good deal, though not all, of the infuriatingly frustrating "random esri error for who knows what reason" issues have been resolved, and continue to improve.
I’m slowly making a map of my back yard in QGIS. One annoyance: I live in a place where tectonic movement over my expected lifespan is significant on the scale of my map. And so are the supposed inaccuracies in various coordinate systems. (I want to record where things are so I can find them again without digging big holes!)
As far as I can tell, QGIS has no particular understanding of either a coordinate relative to the (moving!) crust or of a coordinate in space-time that can be projected to space at future or past times. Surely this should be a thing!
I found HTDP, a web tool that can shift coordinates forward and back in time:
(I have an RTK-capable GPS and NTRIP data via UNAVCO from a nearby CORS station. I was hoping that storing position relative to such-and-such CORS station as it was on such—and-such date would be straightforwardly doable in QGIS.)
Wow that’s a pretty interesting (and scary) challenge.
I know that the coordinate systems get revised every few years. In Australia we used to use AGD83 then it was revised to GDA94 and more recently to GDA2020 — and this was to account for tectonic shifts. And there’s calculations for transforming from one to the other.
That’s the only clues I have on that side of things and I’ve forgotten most of what I’ve known.
Also there’s a concept called “rubber sheeting” for transforming points where you know the amount of error at a bunch of points and you want to transform all the other points by interpolating how much error there would be at those points. The might be useful too, if you’re implementing a solution yourself.
I didn’t find anything in QGIS to give a layer a CRS and an associated date, so I’m not sure how coordinate systems like this are expected to work properly.
I am doing the same, Mainly I wanted to record where pipes and such are located but might as well record buildings, fences and trees while there. I was not worried about geologic movement but all the global coordinate systems felt very clunky when locally mapping at the centimeter level, I finally settled on having a well defined surveyed reference point, and using meters north and east off that.
Perhaps that would work for you. because while the plate is shifting with respect to the globe, everything on that plate will maintain the same relative position.
I set up my own NTRIP base station at a fixed point in the middle of my roof with a hefty bracket, did a PPP survey to determine its location once, and am considering everything relative to that. This spring when I'm back at the surveying, I'll likely do another PPP run and make sure it hasn't moved too much. If it has, I guess I'll have to figure out how to reconcile that. From what I've gathered, I don't think this is too far off from how real surveying works.
I didn't see much reason to use a different unit than degrees. Although while I haven't gotten super in depth learning QGIS, it feels like there's an impedance mismatch in that it seems to be a 2D program that treats degrees as linear units (and then applies a fudge factor to degrees longitude), rather than a native 3D program. So I'm doing all of my collection, point storage, and calculation with scripts outside of QGIS, and then only pushing the cooked results to QGIS for visualization.
I don’t have a good sense for how far one needs to be from one’s NTRIP base station, but I’m pretty happy with the UNAVCO data I’m getting. The nearest station is operated by Serious Professionals, and it won’t move if I mess around with my roof. :)
I've got a professional base station that's decently close, but still far enough to degrade the accuracy a bit (I forget the exact figure but you can find it if you search). When I used the professional base station to figure out the point and compared it to what PPP had found, it seemed to be within that amount of error.
I chose the part of the roof least likely to be disrupted. But if I do need to move it, then I'll have to figure out exactly how I want to map to points going forward.
Instead of a reference + north/east meters, how about two reference points, and then everything is referenced as polar coordinates from there, point A to point B being 0 degrees. My concern would be if the direction of north/east changed over time from the plate movement.
Oh very, very much yes. I've been using QGIS for about five years now as the first step in doing any sort of visualization work around GIS data. Better to trust a GIS application than my own bad d3 code :). It's also been absolutely fantastic for changing projections, changing resolutions, merging GIS files, things like that.
My only complaint about it is that it can be a little bit slow sometimes. It has to redraw everything if you zoom in. That's especially annoying if you're working with data that has millions of multilines or points. But, as a prototyping engine, that's kind of expected and still better than paying for ArcGIS.
Our project is not big enough to justify tools like ArcGIS. Previously I'd used Global Mapper, which is also commercial but is somewhat more reasonably priced.
But I recently gave QGIS a shot and am very happy with what it can do. And the price is right!
I used QGIS to help me import some terrain into Minecraft. I found a lidar elevation map, used QGIS to visualize it in 3d and tweak some things, and then I was able to use another tool to do the conversion into Minecraft. Definitely an impressive tool.
Every field has its own “occupational hazards” - I found that with maps and mapping the maps themselves were so fascinating, you could get drawn in and lose a lot of time just… exploring I guess.
QGis is great, lots of tools are nice. Generally ESRI sucks up all the oxygen and leaves not enough for innovation and implementations from anyone else. There are a lot of features and capabilities needed in the Geospatial world that do not have reliable, rock-solid open source implementations. (I could’ve provided examples a few years ago but now I’ve forgotten the important details and only have opinions left)
I've been using QGIS a bit for simple things. Viewing OSM road data in a PostGIS database. Viewing geojson files. Everything has been more difficult than I expected, but I think that might just come with the territory. I'm working through PostGIS in Action to try to get the requisite background knowledge.
One thing that's frustrated me, and I want to make sure it isn't a misconfiguration on my end, is that QGIS feels really slow. For example, I have a 150MB geojson file which has 300,000 points with associated metadata. Even when I'm zoomed in such that I can only see a few thousand points at a time, if I pan the map over by 50% it takes at least 10 seconds before it loads in the new points. Many long operations seem to take place synchronously on the UI thread, so the whole app is unresponsive while they take place. Clicking the drop down arrow on a PostGIS Schema to view the tables spins for several minutes. No other Postgres tool I have takes that long, so it's not the database. The PostGIS import/export tools were also extremely slow and didn't have progress bars. I'm using 3.24 currently. I don't want to rag on it too hard, but it's really hampered my enthusiasm for working with maps and GIS.
You'd probably be well-served by exporting to some format with a spatial index. GeoJson is just a textfile, not much better than reading in a CSV. The current recommended format is Geopackage (gpkg), which is based on SQLite.
Edit: agreed on the slowness of DB connections, I've found that too, including for relatively small locally-hosted DBs.
Yeah, converting it to gpkg sped it up a lot, thanks!
I assumed that since geojson is completely unsuitable for querying directly, it would load the whole thing into a native in-memory format, but perhaps not.
I don't think geojson is a great format for anything with more than a few MB of data.
I wanted to see exactly how bad it is with a largeish datasets, so I exported the New Zealand address dataset[1] with ~2.5M points as a geopackage (750MB) QGIS loads this fine its a little slow when viewing the entire country but when zoomed into city level it is almost instant to pan around.
Using ogr2ogr I converted it to ndgeojson (2.5GB), It crashed my QGIS while trying to load it. Using shuf I created a random 100,000 points geojson (~110MB) it was unbearably slow in QGIS while panning around 5+ seconds.
I currently use and recommend flatgeobuf[2] for most of my working datasets as it is super quick and doesn't need sqlite to read (eg in a browser).
It is also super easy to convert to/from with ogr2ogr
QGIS is always a little more difficult than expected. Even as someone with a masters in GIS, its still hard for me to remember where they put things or how to do it. Its extremely powerful but can be extremely frustrating too.
GeoJSON is unfortunately one of the worst formats to keep the data you are using in. Its great for transport and interoperability, but there is no way to index the data in the format. It doesn't matter if you are only drawing a few thousand points, it is still looking through all of them to see which ones to draw.
GIS is awesome, and don't let one tool get you down. If its something interesting to you, there is a lot more to it than just QGIS. Part of my pain with QGIS is because I rarely ever use it so I forget what I learned last time. I spend most of my time purely in python, and don't really need to.
No, you're not victim of misconfiguration. QGIS is very slow with large datasets, no matter what source you use (even if you have Oracle with costly Spatial extensions). It is due to some unfortunate design decisions - the app always loads the full dataset from the source, and draws/shaws every single feature. It is possible (through some recent, obscure check in the prefs) to tell it to skip smaller objects when putting everything on the screen, but generally is not possible to tell it "do not load features from PostGIS that are going to render to only few pixels".
Another reason is that the underlying C++ engine is tied in such way with the fairly-large Python codebase, that it seldom uses all CPU cores. so you have a single process app. Add to this the fact that the GPU is not used extensively and you get one very slow APP.
So, yeah, for datasets larger than 50k points its slow on arbitrary hardware. Some analysis are impossible to run unless you go to PostGIS
To be honest, ESRI is not a company that everyone loves, but they really invest into ArcGIS Pro, even though they also are not there yet. Both QGIS and ESRI shy away from spatial SQL which is like times more effective for spatial analysis.
To be honest, QGIS is quite old-school in design, and the core devs know that it would take enormous effort to implement it from scratch... at least the slow parts. The fact that it works and is open source, does the fundamental things you need does not make it the _top_ software. this "opensource ftw" is really stupid when you have to work with hundreds of layers with hundred Ks of points.
Make sure you have estimated table metadata turned on, otherwise QGIS will run a bunch of queries to understand your tables I believe.
We typically use QGIS as a viewing engine only, if you let postgis do the heavy lifting it's a beautiful setup, especially with a tuned dB and a indexed clustered postgres table
> Make sure you have estimated table metadata turned on, otherwise QGIS will run a bunch of queries to understand your tables I believe.
That fixed it, thanks! Looking at the manual, there is a tip that tells you to turn that on, otherwise it will read the entire table to characterize the geometries... It's a bit mad that the default behavior has it querying potentially gigabytes of information across the network every time you open the app and click a dropdown, with no progress bar. But it's definitely the type of app where you need to read the manual, and it says so right there.
182 comments
[ 3.2 ms ] story [ 237 ms ] threadQGIS also has server and headless options, plus lots of the QGIS core is importable into python scripts.
From what I understood, qgis was doing its job properly. It was a neat tool. I guess the whole stack for this use case was just too fragile.
I use it daily, and frequently, for all sorts of work building a location API SaaS company. It's a sterling success of open source, on par with as impactful as Linux, for the geospatial world (IMO).
I tried to perform what I thought were basic tasks of drawing a few line segments and measuring geodesic distances between two points, but I needed a janky plugin for the former and never figured out how to do the latter.
I really do want to learn it however, since it's a lot easier sometimes to work interactively than doing everything in Python code, and I know QGIS is supposed to be a very powerful tool in general.
- The QGIS Training Manual is good. Not great; don't be afraid to go out of order after the first few modules to get to parts useful to you, because the module organization after "Creating Vector Data" stops being terribly linear. https://docs.qgis.org/3.22/en/docs/training_manual/index.htm...
- https://www.qgistutorials.com/en/ is more project- and task-oriented, and where I learned georeferencing and digitizing (drawing over raster maps)
- If you prefer video, https://www.youtube.com/watch?v=NHolzMgaqwE is geared more toward using third-party data
EDIT: There's also this UCDavis workshop for people with no mapping experience. Haven't used it but at a glance it looks pretty comprehensive. https://github.com/ucdavisdatalab/Intro-to-Desktop-GIS-with-..., https://www.youtube.com/watch?v=PAcZUPSmqDE
That being said, as I've outlined in the article, I certainly haven't properly learned it...
But having dabbled in Python/Folium and the like (most recently did geo vis with Superset for another article), some of the datasets and functionalities were familiar (I also had to do point-in-polygon problems in Python, way back when, so maybe that mental scar helped :) ). I found random blog articles and stackexchange posts for specific questions I got stuck on the most helpful.
For instance, I have this link saved: https://gis.stackexchange.com/questions/421467/not-geotagged... , which got me down the rabbit hole of "Oh, you can make a style a function of other things? You can read EXIF? How neat!" (where I then found some other posts).
What I will say, almost everything online is out of date - the GUI changed frequently, plugins are now native functionalities etc. The HTML tooltip thing I did is also a modification of an older blog article (which I'm afraid I didn't save), whereas the contour line styling I took verbatim from the article I linked in that paragraph.
GIS is a huge space, and you could specialize in one niche for years or go more general, so it's really based on what you want to do.
I'd say there's three or four general areas: mapping & map data (mostly focused on visualization), geocoding & search (focused on helping people find things in natural language, think text -> coordinates, text -> point of interest -> coordinate), routing (focused on the mechanics of getting things from point a to point b efficiently), and finally a huge, diverse bucket of "other location APIs" (anything from geofencing to timezones to ETL specialization).
I'm happy to chat about this. We aren't hiring, but there are lots of companies that are.
QGIS is great software, and I respect what they're doing.
ArcGIS Online is a pretty great option and the Living Atlas [1] makes it really easy to consume a lot of data (a bunch of NASA datasets were recently added) and make cool maps. ArcGIS Online ends up being almost like Dropbox for geographic data, but instead of just files you can get full services, and a suite of apps for taking advantage of the services. That includes taking maps offline, making field edits, and syncing them (although that starts to cost money).
You can try the map tool without signing in, and that gives you a good idea of what you can do in the browser (custom popups, charts, tables, filtering, symbology, clustering, cartographic blending, etc). https://www.arcgis.com/apps/mapviewer/index.html?layers=dece...
The personal use subscription is $100/year and includes the full desktop software and cloud storage. [2]
[1] https://livingatlas.arcgis.com/en/browse/#d=2 [2] https://www.esri.com/en-us/arcgis/products/arcgis-for-person...
edit: forgot to mention there is a great developer story: https://developers.arcgis.com/javascript/latest/sample-code/
Are there any plans for a native macOS version? For someone that wants to learn more about GIS in general, does the web version have what is needed?
In my personal experience with using ArcGIS, I have only needed Pro in two situations. The first is when I'm trying to georeference a raster (I just bought one from SkyFi). The second is when I've bought a tutorial book that is based around ArcGIS Pro (e.g. Modern Policing Using ArcGIS Pro).
If you want to learn ArcGIS and not rely on Pro, you can visit the learn site and filter to just ArcGIS Online: https://learn.arcgis.com/en/gallery/#?p=agol
I won't say more, but I have personally been impressed by the rate of improvement in feature capability in ArcGIS Online. I am not aware of any plans to move Pro to macOS, I feel your pain there.
There are currently no plans to rewrite ArcgGIS pro for M1 mac hardware according to ESRI. The move to Pro should have made it much easier to do, but its still a huge task.
The ESRI community have recently had a lot of success with M1/2 hardware, Parallels 17+, Windows 11 arm, and ArcGIS Pro. Just note that you won't be making 3d videos with it comfortably. Everything else seems to be working very well.
Their desktop version doesn't. Their target audience is large to very large enterprise. Their customer base is overwhelmingly windows based. They know this, so they've shrunk their dev costs to account for it.
ArcGIS then proceeds to apply symbology one layer at a time, taking between 30 to 75 seconds per layer to apply it.
In QGIS, right click in layer, copy symbology, select other layers right click, paste symbology. Done in 3 seconds.
And like that I could be hours and hours talking about similar experiences.
ArcGIS might be much more powerful, but honestly it isn't worth spending 45 minutes to what in QGIS can be done in 5 minutes.
This post doesn’t mention ArcGIS at all, except for as a data source that your county government probably uses.
I remember when I didn’t know about ArcGIS and thought it was cool to build apps that put points on MapBox maps - which I later found out is a waste of time, because ArcGIS already does that job well.
I think this post leaves people with the impression that GIS is niche, or difficult, or not accessible to lay users. My personal opinion is that recent advancements in Web GIS make the field about as accessible to people as things like word processing, while QGIS is stuck in the ArcMap days, catering exclusively to trained GIS professionals.
It is absolutely meaningful to point out that you don’t need to be a trained data engineer to do this kind of work, and that this 4,000 word journey could have been done in 10 minutes of clicking around user-friendly software.
One of my favorite experiences from that era: We were meeting with a few ESRI reps for some integration stuff. The lead hot-shot was on his phone playing around during the meeting. He was basically on autopilot. The other two folks were working with GeoJSON response convertors. I said, "I built one of those with TopoJSON". One guy said, "I've never heard of it". I showed them how it was much more efficient and used splines instead of points. The lead dropped his phone and said, "I need you to tell me MORE about that". I showed them the service. They invited me to lunch, I politely declined and said, "today's my last day so I have a ton of things to wrap up". I do miss that realm sometimes.
I've had difficulty making it work for production services (especially extensions). Ultimately moved more towards the Python/OSM stack.
Such a fun space.
https://blog.cleverelephant.ca/2018/11/esri-dominates.html
Ed. "deployed"/"deployment" 0400GMT
I've been telling people for years they should be using GIMP and Linux but nobody cares. At least Oracle's stranglehold seems to be weakening.
The alternative to Photoshop is that I gave $40 to Pixelmator and now I have an awesome photo editor that does plenty.
Yeah, there's a massive amount of essential features just missing from OSM and QGIS. Like, even the most basic - layers. They don't even really exist in OSM side of things. Whereas if I'm doing travel-time-bands from medical centers, I absolutely must use multiple boundary layers.
Then again, I'm an engineer, not a data scientist. My colleagues (all data scientists) do all sorts of crazy awesome work. Again, unless you're hacking R/python code and piecemeal it together, you're using ArcGIS. Then again, "ArcGIS Notebook" is a dockerized Jupiter system for doing just that in ArcGIS enterprise.
edit: As for me personally, since I'm working in geoinformatics, I've been teaching myself the hows and whats. Im pretty proficient at both ArcGIS and QGis, as I'm trying to learn them side by side.
If I am doing my own projects, I will use QGis. That's not even a discussion. I'm not about to use proprietary systems to lock up my data.
Lets just say the data intersection of: density and geocoding of users, density and location of office sites, travel time bands, and service level expectations all have a part in the final calculation.
The data is either base layers imported, or calculated layers. There is no one "comprehensive layer" nor does that concept make sense. Different groups handle their own data. We import it in and use their data for our final result.
(In fact, its the OSM approach of everything as 1 layer and tags everywhere is why a DB import and data handling is so obnoxious.)
And how would that concept even apply to postgis? A postgis query can operate over many tables.
And there are a variety of open source tools that allow multi-dimensional data structures. That’s multiple layers in multiple dimensions.
So I’m hoping, if there’s some aspect of a GIS layer I don’t understand, he can explain it. Qgis totally supports multiple layers in the sense that most people think of it.
He doesn’t have to divulge “secret information” to do this. If he is referring to some aspect of the qgis architecture he can just say so. Qgis is open source, their GitHub repo is clean and well documented. Im merely asking for him to explain why it handling multiple layers is somehow not really handling multiple layers.
I worked in a GIS Lab with an ESRI endorsed all Arc stuff textbook author/instructor for a number years. Working on similar projects, for anything remotely complicated, I could implement anything he could do, with some python and PostGIS SQL, usually faster, and instantly reproducible.
ArcGIS is a crutch for people who can’t and aren’t willing to program. This goes for all data manipulation tools that aren’t focused on visualization.
If you have an ETL pipeline, probably best to get GIT involved.
Whether or not they are a good company that provides quality, unparalleled products is not a point I’m contending. They are not competing on that alone. That’s my point.
I think Esri is (and has been) in a very similar position to Microsoft's in the late 1990s -- having achieved market dominance, they feel like open source software is the biggest threat to their business. But I think the presence of QGIS is creating competition, which is nothing but good for the industry.
(I worked as a developer for Esri for 15 years)
Also, the GIS space tends to have crossover between different departments in companies. You have a mix of IT departments, GIS specialists outside of IT departments managing infrastructure, data management, casual users etc. Each has a different budget and a different competitive environment. ESRI sits as a compromise between these different groups. Each group may be able to adopt a competitor but not gain traction outside their group.
I was on a call to ask some ESRI rep to add some labeled points to my client's existing map tool. It was a somewhat surreal, weird experience where I got the feeling they were making the work seem much more difficult than it was. Their estimate turned out to insanely off the (my) mark, at eye-watering hourly rates.
At first I thought they had enough business and didn't really care about us. But reading this thread, it seems I was wrong. They are the Oracle of the GIS industry.
I've spent a large number of hours trying to fight ESRI software, lost more fights than I've won and even my victories have mostly been pyrrhic.
^ For instance, a really neat one that renders ocean features in as much detail as the land typically gets
Assuming the service ever even gets to 2M tile requests/month. That alone would be quite the accomplishment.
Though with Gaia going to hell, something new needs to come along.
Have an AI crawl a RASTER dataset and VECTORIZE all lines into SVG.
We make 3D Maps of American Landscapes in bronze.
We take Digital Elevation Model (DEM) data, do light transformations in QGIS and convert it to an .STL file before additional 3D modeling.
Our latest project was a hairy one doing Oahu (https://terramano.co/blogs/product/oahu-bronze-3d-map)
https://www.usgs.gov/the-national-map-data-delivery/gis-data...
They have full US coverage and many infill sets at higher resolutions.
They have 1 metre DEMs ( ~ one elevation per three foot x three foot square )
https://www.sciencebase.gov/catalog/items?q=&filter=tags=Dig...
and more if you get to know their community and products (they are a firehose of likely more data than many can afford to reliably store).
What resolution? Totally depends on the area the customer would like to cover. If it’s their ranch or property, we usually need 1-meter. If it’s a mountain range than 30-Meter works.
It mainly depends on the resolution limit for 3D printing. So it also depends on the size of the model they want.
Unfortunately not all areas are covered with high res
Full multi channel or just LIDAR?
https://apollomapping.com/digital-elevation-models https://www.l3harrisgeospatial.com/Data-Imagery/Elevation-Da...
I don't wanna share the number but it was 6 figures
(Near) global coverage, 90m resolution. Easy to fetch tiles with a script.
If you have weird artifacts, I'm guessing that is due to the underlying data vs QGIS itself. Have you looked at their documentation (https://demto3d.com/en/)?
I outline how the whole process works here https://www.gregkamradt.com/gregkamradt/2020/2/29/manufactur...
Do you have a video link of what you're referring to?
I once tried to use the molds to make chocolate representations of the mountains ha! I learned the hard way that tempering is difficult for a novice
You have fewer products for sale than I was expecting.
I live in NE Los Angeles, which has the Verdugo and San Gabriel mtns, plus Mt Washington and other modest peaks - I think it would look great in this style. Especially because all the development would be excluded.
https://docs.google.com/document/d/1IkiHG_Z5JS03mWYHv-KNAhi8...
edit: whoops added link
For every new location you do the process looks like: 1. Get the data and prep it for print (fixed) 2. 3D print it (fixed) 3. Rubber Mold (fixed) 4. Wax Model (variable) 5. Bronze (variable)
Steps 1-3 are 40-60% of the costs. So I haven't put the money out of pocket yet to put up new locations. I've let customer's ask first and then do them.
Surprisingly, most of our orders have been custom
Here's my info packet on the custom process https://docs.google.com/document/d/1IkiHG_Z5JS03mWYHv-KNAhi8...
Anyway they look beautiful.
A couple of packages used by qgis worth calling out - postgis which really grew up with qgis, and pdal which is a relatively new but incredible package for working with point cloud data.
I think a simplified GIS program with stripped-down functionality and an intuitive UI could be a big hit. Think of SketchUp versus SolidWorks or ProE.
Here's a couple maps that I have made about the Honoulu marathon that I wouldn't have made in a more complex/time-intensive piece/powerful piece of software: - https://felt.com/map/UNOFFICIAL-Honolulu-Marathon-2022-TCg9C... - https://felt.com/map/UNOFFICIAL-Honolulu-Marathon-2022-Road-...
Or to put it another way, what's the workflow that a simpler, more opinionated interface would solve, and what features could or would you sacrifice for it?
- A lot of people come to it wanting to edit or annotate existing city/street maps with geometry, like you're describing.
- My first encounters with it were trying to plot points on a map to build a heat map, in a data analysis context, and I was coming from Google Earth/KML.
- The next time, I wanted to georeference raster maps to make more accurate vector data, and I was coming from Inkscape and Illustrator.
- Another popular use case is to work with topographical and height data, which is another workflow with mostly another interface in QGIS.
- And all of these can have very different output goals - slippy maps with raster or vector tiles, topo maps, 3D renderings and prints, GeoJSON and PostGIS data.
The "most obvious" app for each of those use cases would each be different, with different interfaces. And I can see where that would be good for beginners, or for limited use cases, but at some point each of those use cases intersects with another. Drawing bike lanes on a map quickly means working with topo data. Georeferencing a new base map eventually hits plotting POIs and drawing things like political borders as geometry. Outputting a heat map from data you plot in QGIS can work, but then you need to plug it into a database at some point so the data can be updated in a shared or automated context.
IMO what would be more useful for QGIS is a ground-up, no-assumptions review of its UI similar to what Blender did in 2.8/3.0. Find where they can consolidate redundant or conflicting tools and interfaces, redesign their mouse/keyboard interactions and tool palettes around how users intuit they should work, and provide alternate modes for the more complex workflows that existing users are accustomed to. Blender's redesign proved pretty well that an esoteric UI can be streamlined and modernized without sacrificing power user workflows.
But I don't know if the QGIS UI community is as large, organized, or focused on modern UI design as Blender's, which was a rather unique case in FLOSS. And it'd be difficult to divorce QGIS's GIS data-editing engine from its interface as a whole to build standalone tools with smaller UIs focused on tasks instead of toolboxes.
Plugins might be able to do this, which could be an interesting approach. Instead of adding features like plugins tend to do, one could ship a "flavor" of QGIS with pre-installed and -configured plugins that provide narrower, task-focused interfaces around existing features.
These two already dominate more casual mapping
You likely either: - needed to export an image of the data (raster); - needed to digitise the data within; - needed to extract the vector data first from the pdf and work out how to deal with it.
Opening a pdf without appropriate data structures into a gis software package is akin to taking a picture of a billboard that has a printed image of a map on it and then expecting to be able to do anything with that picture. The pdf is a bit better, but not by much unless it began life as a georeferenced pdf with vector data maintained.
By the time I got it, it was just a vector image.
The infrastructure in question was pipelines, represented in the PDF by lines, so georeferencing them was enough to get spatial data adequate for the application.
https://blog.qgis.org/2023/01/16/crowd-funding-call-2023/
I'd kill to find a full remote job doing that.
Where do people with QGIS knowledge hang out (especially devs)?
I'm saddened that complex-number polynomial expansion fell out of vogue and fear that was the last wall that kept the muggles out of hard core map projections.
Then, about a year ago, after several years away from the scene, I got some mojo back and started investigating modern QGIS. I was very pleasantly surprised, it immediately felt like an upgrade from what I used to use. And also felt like coming home, in that it was obvious and intuitive how to accomplish the standard functions. If someone had put me in (that famous thing) a blindfolded time machine for a zero-year trip, popped me out, put me in front of QGIS and said "Welcome! This is ArcGIS 9.6!", it wouldn't have been at all unbelievable. If you know what I mean.
No it's not perfect. Casually and hastily wing your way around a project, copypasting here and there, setting up all kinds of parameters and advanced display bits&bobs, and - like most complex geospatial software - you'll, once in awhile, find yourself unceremoniously dumped out onto the desktop with a software error. So a highly robust and restore-able workflow is still essential. But regardless it's up there with the very best packages I've ever used.
Importantly, a good deal, though not all, of the infuriatingly frustrating "random esri error for who knows what reason" issues have been resolved, and continue to improve.
As far as I can tell, QGIS has no particular understanding of either a coordinate relative to the (moving!) crust or of a coordinate in space-time that can be projected to space at future or past times. Surely this should be a thing!
I found HTDP, a web tool that can shift coordinates forward and back in time:
https://www.ngs.noaa.gov/TOOLS/Htdp/Htdp.shtml
And I found this discussion:
https://www.gpsworld.com/the-effects-of-tectonic-plate-movem...
But I haven’t found anything easy to use.
(I have an RTK-capable GPS and NTRIP data via UNAVCO from a nearby CORS station. I was hoping that storing position relative to such-and-such CORS station as it was on such—and-such date would be straightforwardly doable in QGIS.)
I know that the coordinate systems get revised every few years. In Australia we used to use AGD83 then it was revised to GDA94 and more recently to GDA2020 — and this was to account for tectonic shifts. And there’s calculations for transforming from one to the other.
That’s the only clues I have on that side of things and I’ve forgotten most of what I’ve known.
Alternatively: right click layer > save as, specify in properties.
There are geoprocessing tools to do the same.
Sometimes the input data may not be suitable for setting a CRS.
Perhaps that would work for you. because while the plate is shifting with respect to the globe, everything on that plate will maintain the same relative position.
I set up my own NTRIP base station at a fixed point in the middle of my roof with a hefty bracket, did a PPP survey to determine its location once, and am considering everything relative to that. This spring when I'm back at the surveying, I'll likely do another PPP run and make sure it hasn't moved too much. If it has, I guess I'll have to figure out how to reconcile that. From what I've gathered, I don't think this is too far off from how real surveying works.
I didn't see much reason to use a different unit than degrees. Although while I haven't gotten super in depth learning QGIS, it feels like there's an impedance mismatch in that it seems to be a 2D program that treats degrees as linear units (and then applies a fudge factor to degrees longitude), rather than a native 3D program. So I'm doing all of my collection, point storage, and calculation with scripts outside of QGIS, and then only pushing the cooked results to QGIS for visualization.
I chose the part of the roof least likely to be disrupted. But if I do need to move it, then I'll have to figure out exactly how I want to map to points going forward.
My only complaint about it is that it can be a little bit slow sometimes. It has to redraw everything if you zoom in. That's especially annoying if you're working with data that has millions of multilines or points. But, as a prototyping engine, that's kind of expected and still better than paying for ArcGIS.
But I recently gave QGIS a shot and am very happy with what it can do. And the price is right!
QGis is great, lots of tools are nice. Generally ESRI sucks up all the oxygen and leaves not enough for innovation and implementations from anyone else. There are a lot of features and capabilities needed in the Geospatial world that do not have reliable, rock-solid open source implementations. (I could’ve provided examples a few years ago but now I’ve forgotten the important details and only have opinions left)
Can you list a few please?
Is the author assuming you're either from the US or European if you're reading the blog?
I’m originally from Germany, hence the bias towards readers from Europe. :)
One thing that's frustrated me, and I want to make sure it isn't a misconfiguration on my end, is that QGIS feels really slow. For example, I have a 150MB geojson file which has 300,000 points with associated metadata. Even when I'm zoomed in such that I can only see a few thousand points at a time, if I pan the map over by 50% it takes at least 10 seconds before it loads in the new points. Many long operations seem to take place synchronously on the UI thread, so the whole app is unresponsive while they take place. Clicking the drop down arrow on a PostGIS Schema to view the tables spins for several minutes. No other Postgres tool I have takes that long, so it's not the database. The PostGIS import/export tools were also extremely slow and didn't have progress bars. I'm using 3.24 currently. I don't want to rag on it too hard, but it's really hampered my enthusiasm for working with maps and GIS.
Edit: agreed on the slowness of DB connections, I've found that too, including for relatively small locally-hosted DBs.
I assumed that since geojson is completely unsuitable for querying directly, it would load the whole thing into a native in-memory format, but perhaps not.
I wanted to see exactly how bad it is with a largeish datasets, so I exported the New Zealand address dataset[1] with ~2.5M points as a geopackage (750MB) QGIS loads this fine its a little slow when viewing the entire country but when zoomed into city level it is almost instant to pan around.
Using ogr2ogr I converted it to ndgeojson (2.5GB), It crashed my QGIS while trying to load it. Using shuf I created a random 100,000 points geojson (~110MB) it was unbearably slow in QGIS while panning around 5+ seconds.
I currently use and recommend flatgeobuf[2] for most of my working datasets as it is super quick and doesn't need sqlite to read (eg in a browser).
It is also super easy to convert to/from with ogr2ogr
ogr2ogr -f flatgeobuf output.fgb input.geojson
[1] https://data.linz.govt.nz/layer/105689-nz-addresses/data/ [2] https://github.com/flatgeobuf/flatgeobuf
GeoJSON is unfortunately one of the worst formats to keep the data you are using in. Its great for transport and interoperability, but there is no way to index the data in the format. It doesn't matter if you are only drawing a few thousand points, it is still looking through all of them to see which ones to draw.
GIS is awesome, and don't let one tool get you down. If its something interesting to you, there is a lot more to it than just QGIS. Part of my pain with QGIS is because I rarely ever use it so I forget what I learned last time. I spend most of my time purely in python, and don't really need to.
Another reason is that the underlying C++ engine is tied in such way with the fairly-large Python codebase, that it seldom uses all CPU cores. so you have a single process app. Add to this the fact that the GPU is not used extensively and you get one very slow APP.
So, yeah, for datasets larger than 50k points its slow on arbitrary hardware. Some analysis are impossible to run unless you go to PostGIS
To be honest, ESRI is not a company that everyone loves, but they really invest into ArcGIS Pro, even though they also are not there yet. Both QGIS and ESRI shy away from spatial SQL which is like times more effective for spatial analysis.
To be honest, QGIS is quite old-school in design, and the core devs know that it would take enormous effort to implement it from scratch... at least the slow parts. The fact that it works and is open source, does the fundamental things you need does not make it the _top_ software. this "opensource ftw" is really stupid when you have to work with hundreds of layers with hundred Ks of points.
We typically use QGIS as a viewing engine only, if you let postgis do the heavy lifting it's a beautiful setup, especially with a tuned dB and a indexed clustered postgres table
That fixed it, thanks! Looking at the manual, there is a tip that tells you to turn that on, otherwise it will read the entire table to characterize the geometries... It's a bit mad that the default behavior has it querying potentially gigabytes of information across the network every time you open the app and click a dropdown, with no progress bar. But it's definitely the type of app where you need to read the manual, and it says so right there.
It should only do it once per table but if you are dropping and creating tables often it will be making the DB have a heavy workload
I think theirs an assumption within the geo community of read heavy workloads but I could be wrong