Hiya, I'm the original author. tl;dr for those deciding whether or not to read it: If you are thinking about build versus buy for your ETL solution, in this day and age, there are enough great tools out there where buy…
Rushing a job search. Decided it was time to move on (right decision), but allowed one company to court me and did not take enough time to play the field (bad decision).
The next step is to make the tool easy enough to use such that GPT-3 could build a connector in 2 hours.
We couldn’t agree more that producing high-quality connectors requires a lot of work. The hardest part about this task is that connectors must evolve quickly (due to changes in the API, new corner cases, etc). The…
If you are interested, John (one of the co-founders) wrote an article about how we are imagining ELT evolving. https://airbyte.io/articles/data-engineering-thoughts/why-th...
(Airbyte Engineer) I think what you're saying here is often true until it isn't. For a personal project where you're pulling data from one API? Sure. Once you have an engineering system with multiple engineers relying…
Different Airbyte engineer here! Wanted to help answer your question as to what "optional normalized schemas" means. When writing data into your data warehouse we provide 2 options: 1. write each record as a json blob.…
Hiya, I'm the original author. tl;dr for those deciding whether or not to read it: If you are thinking about build versus buy for your ETL solution, in this day and age, there are enough great tools out there where buy…
Rushing a job search. Decided it was time to move on (right decision), but allowed one company to court me and did not take enough time to play the field (bad decision).
The next step is to make the tool easy enough to use such that GPT-3 could build a connector in 2 hours.
We couldn’t agree more that producing high-quality connectors requires a lot of work. The hardest part about this task is that connectors must evolve quickly (due to changes in the API, new corner cases, etc). The…
If you are interested, John (one of the co-founders) wrote an article about how we are imagining ELT evolving. https://airbyte.io/articles/data-engineering-thoughts/why-th...
(Airbyte Engineer) I think what you're saying here is often true until it isn't. For a personal project where you're pulling data from one API? Sure. Once you have an engineering system with multiple engineers relying…
Different Airbyte engineer here! Wanted to help answer your question as to what "optional normalized schemas" means. When writing data into your data warehouse we provide 2 options: 1. write each record as a json blob.…