Airflow is far from perfect, but I don't understand your concerns. I work in a big and messy company and even messier department. We have jobs running in Databricks, Snowflake, sometimes we read data from API end…
> if you want to convert some scheduled pipeline to some event-driven architecture Airflow has sensors and triggers. https://airflow.apache.org/docs/apache-airflow/stable/author... But in the core it is built around…
There is nice summary on the topic: https://aws.amazon.com/blogs/big-data/choosing-an-open-table... ("Optimizing read performance"). Those technologies primary "Data Management at Scale" but they also extend…
from zoneinfo import ZoneInfo import pendulum import datetime def pendulum_to_datetime(pendulum_dt: pendulum.DateTime) -> datetime.datetime: return datetime.datetime.fromtimestamp(pendulum_dt.timestamp(),…
>pendulum is probably your best bet It is best until you work with Pandas DataFrame (https://stackoverflow.com/questions/47849342/making-pandas-w...)
I doubt it was .exe, probably .msi (installer) or zip file.
no sure it is relevant, probably all heavy lifting done in C++ and Rust https://searchfox.org/mozilla-central/search?q=render&path=&...
https://github.com/evolus/pencil
I think the article and most of the commentators are missing a real problem. The real problem is that there is an underlying complexity, which has no simple solution - collaboration of teams/groups of people with…
I am using Java occasionally, there isn't any special learning curve if you know basic syntax. Same as in Python you don't start with Jdango and virtual environments, you shouldn't start using Java with Spring…
Term "toxic" is toxic by itself. Periods when engineer don't understand the problem should be spent on analysis of the problem domain. "Now I am working on defining the problem domain" - is an activity to work "I don't…
Talend Studio? https://www.talend.com/products/talend-open-studio/
Apache Airflow is tool for *analytical data* pipelines. If you want to create a daily/monthly data feed from your operational DB, move to it into Data warehouse, run jobs to process/aggregate that data and create…
he has "a wonderful secretary who looks at the incoming mail"
People like [Sam Walton](https://en.wikipedia.org/wiki/Sam_Walton) born in poor farmer family or [Jeff Bezos](https://en.wikipedia.org/wiki/Jeff_Bezos) son of 17 year old high school student mother and 19 years old…
do you have doubts those people are skilled? https://en.wikipedia.org/wiki/Brad_Fitzpatrick https://news.ycombinator.com/item?id=21727925
there is a strict mode https://sqlite.org/src/wiki?name=StrictMode
> No Java I am not sure what "essential tool" means if pandoc included in it. For me Intellij, Eclipse, Dbeaver are more essential (+ few other Specific tools based on Eclipse). C# - I am not sure, MS Office?
> Python Design Patterns My brain puts it next to: globe of China, Jewish Olympic games (real thing).
I have mixed feeling about Nim. I've tried to learn it few years ago. It was "sold" like a Python replacement, it looks like a Python a bit, but when you write code it feels nothing like python at all. Few things made…
Yes, that part is different, reusability of the code is not expected to be the same as normal software code. Not different that the code starts from requirements (which are different) and it should be correct, which…
I don't like the code. My start point would be that import pandas as pd import numpy as np if __name__ == "__main__": n_samples = 10000 samples_np = pd.DataFrame(np.random.randint(1, 7, n_samples),…
For a bigger code base projects fixed typing contributes to speed of development. Add to it dependency management you will have clear win-win for something like Java over Python.
Talend is sold sort of well (ex. https://help.talend.com/r/mjoDghHoMPI0yuyZ83a13Q/YYVSsDiyJ3v...)
What about running those apps on Android and iPhones?
Airflow is far from perfect, but I don't understand your concerns. I work in a big and messy company and even messier department. We have jobs running in Databricks, Snowflake, sometimes we read data from API end…
> if you want to convert some scheduled pipeline to some event-driven architecture Airflow has sensors and triggers. https://airflow.apache.org/docs/apache-airflow/stable/author... But in the core it is built around…
There is nice summary on the topic: https://aws.amazon.com/blogs/big-data/choosing-an-open-table... ("Optimizing read performance"). Those technologies primary "Data Management at Scale" but they also extend…
from zoneinfo import ZoneInfo import pendulum import datetime def pendulum_to_datetime(pendulum_dt: pendulum.DateTime) -> datetime.datetime: return datetime.datetime.fromtimestamp(pendulum_dt.timestamp(),…
>pendulum is probably your best bet It is best until you work with Pandas DataFrame (https://stackoverflow.com/questions/47849342/making-pandas-w...)
I doubt it was .exe, probably .msi (installer) or zip file.
no sure it is relevant, probably all heavy lifting done in C++ and Rust https://searchfox.org/mozilla-central/search?q=render&path=&...
https://github.com/evolus/pencil
I think the article and most of the commentators are missing a real problem. The real problem is that there is an underlying complexity, which has no simple solution - collaboration of teams/groups of people with…
I am using Java occasionally, there isn't any special learning curve if you know basic syntax. Same as in Python you don't start with Jdango and virtual environments, you shouldn't start using Java with Spring…
Term "toxic" is toxic by itself. Periods when engineer don't understand the problem should be spent on analysis of the problem domain. "Now I am working on defining the problem domain" - is an activity to work "I don't…
Talend Studio? https://www.talend.com/products/talend-open-studio/
Apache Airflow is tool for *analytical data* pipelines. If you want to create a daily/monthly data feed from your operational DB, move to it into Data warehouse, run jobs to process/aggregate that data and create…
he has "a wonderful secretary who looks at the incoming mail"
People like [Sam Walton](https://en.wikipedia.org/wiki/Sam_Walton) born in poor farmer family or [Jeff Bezos](https://en.wikipedia.org/wiki/Jeff_Bezos) son of 17 year old high school student mother and 19 years old…
do you have doubts those people are skilled? https://en.wikipedia.org/wiki/Brad_Fitzpatrick https://news.ycombinator.com/item?id=21727925
there is a strict mode https://sqlite.org/src/wiki?name=StrictMode
> No Java I am not sure what "essential tool" means if pandoc included in it. For me Intellij, Eclipse, Dbeaver are more essential (+ few other Specific tools based on Eclipse). C# - I am not sure, MS Office?
> Python Design Patterns My brain puts it next to: globe of China, Jewish Olympic games (real thing).
I have mixed feeling about Nim. I've tried to learn it few years ago. It was "sold" like a Python replacement, it looks like a Python a bit, but when you write code it feels nothing like python at all. Few things made…
Yes, that part is different, reusability of the code is not expected to be the same as normal software code. Not different that the code starts from requirements (which are different) and it should be correct, which…
I don't like the code. My start point would be that import pandas as pd import numpy as np if __name__ == "__main__": n_samples = 10000 samples_np = pd.DataFrame(np.random.randint(1, 7, n_samples),…
For a bigger code base projects fixed typing contributes to speed of development. Add to it dependency management you will have clear win-win for something like Java over Python.
Talend is sold sort of well (ex. https://help.talend.com/r/mjoDghHoMPI0yuyZ83a13Q/YYVSsDiyJ3v...)
What about running those apps on Android and iPhones?