I think this summarizes the topic quite well: https://pyfound.blogspot.com/2022/05/the-2022-python-languag...
Project A | (Senior) Data Engineer | Lake Bodensee, Austria or Stuttgart, Germany | ONSITE (REMOTE during covid), Full-time | https://www.project-a.com/careers/data-engineer-mfd-49896240... Work where others make…
(Original author here) That setup is of course an option. The point of the article is to not have javascript pixels in the website for tracking, and it's easy to have server side-tracking. So anything that's not based…
Also weird: it's the name of a giant ugly guinea pig: https://en.wikipedia.org/wiki/Mara_(mammal)
GNU Make is indeed the least verbose/ boilerplate-heavy tool and I use it for a lot of things. The problem with Make is lacking acceptance amongst younger programmers who always want to work with the latest technologies.
(author here) Currently there is a hard dependency to Postgres for the bookkeeping tables of mara. I'm working on dockerizing the example project to make the setup easier. For ETL, Mysql, Postgres & SQL Server are…
For example the PyPI download stats pipeline is here: https://github.com/mara/mara-example-project/tree/master/app... The __init__.py contains the pipeline, and the rest is the SQL files that do the transformations
(author here) The mara example project [1] does exactly that. It combines PuPI download stats with Github repo activity data. [1] https://github.com/mara/mara-example-project
(author here) It intentionally doesn't have a scheduler, just definition and parallel execution of pipelines. For scheduling, use Jenkins, cron or Airflow. Currently you can get notifications for failed runs in slack.…
(author here) That's absolutely correct. Mara uses Python's multiprocessing [1] to parallelize pipeline execution [2] on a single node so it doesn't need a distributed task queue. Beyond that (and visualization) it…
(author here) Connection information is configured in code through [1], see [2] for an example. It's very easy to run other workloads. Either by directly invoking Python functions from tasks or by writing own commands…
(author here). Mara data integration is indeed a glorified version of Make (with cost based scheduling and lots of visualizations)
Yes. If you want to have bing aerial images next to google satellite imagery, use this app to display bing maps in google earth: http://ge-map-overlays.appspot.com/bing-maps/aerial
I think this summarizes the topic quite well: https://pyfound.blogspot.com/2022/05/the-2022-python-languag...
Project A | (Senior) Data Engineer | Lake Bodensee, Austria or Stuttgart, Germany | ONSITE (REMOTE during covid), Full-time | https://www.project-a.com/careers/data-engineer-mfd-49896240... Work where others make…
(Original author here) That setup is of course an option. The point of the article is to not have javascript pixels in the website for tracking, and it's easy to have server side-tracking. So anything that's not based…
Also weird: it's the name of a giant ugly guinea pig: https://en.wikipedia.org/wiki/Mara_(mammal)
GNU Make is indeed the least verbose/ boilerplate-heavy tool and I use it for a lot of things. The problem with Make is lacking acceptance amongst younger programmers who always want to work with the latest technologies.
(author here) Currently there is a hard dependency to Postgres for the bookkeeping tables of mara. I'm working on dockerizing the example project to make the setup easier. For ETL, Mysql, Postgres & SQL Server are…
For example the PyPI download stats pipeline is here: https://github.com/mara/mara-example-project/tree/master/app... The __init__.py contains the pipeline, and the rest is the SQL files that do the transformations
(author here) The mara example project [1] does exactly that. It combines PuPI download stats with Github repo activity data. [1] https://github.com/mara/mara-example-project
(author here) It intentionally doesn't have a scheduler, just definition and parallel execution of pipelines. For scheduling, use Jenkins, cron or Airflow. Currently you can get notifications for failed runs in slack.…
(author here) That's absolutely correct. Mara uses Python's multiprocessing [1] to parallelize pipeline execution [2] on a single node so it doesn't need a distributed task queue. Beyond that (and visualization) it…
(author here) Connection information is configured in code through [1], see [2] for an example. It's very easy to run other workloads. Either by directly invoking Python functions from tasks or by writing own commands…
(author here). Mara data integration is indeed a glorified version of Make (with cost based scheduling and lots of visualizations)
Yes. If you want to have bing aerial images next to google satellite imagery, use this app to display bing maps in google earth: http://ge-map-overlays.appspot.com/bing-maps/aerial