Growing up in a somewhat remote part of India, I would tune to BBC, Radio Australia to listen to test cricket commentary, on short wave. I have fond memories and owe a lot of my personal growth to SW.
Thanks for the clarification on this particular point. I was not aware of it.
I don't know all the history, but Magnus comes off as above the game. “I am playing at least one more day here in New York and, if I do well, another day after that,” Really? And I saw another quote that said "f*k you"…
Of late, I have been googling a lot of SAS and may have contributed to its rise in ranking! Not coding in SAS but moving SAS to Python. Speaking of enterprise Java, there's a ton of enterprise SAS and it's moving to…
Not sure why you were downvoted. Fitness, equipment (racket and strings) are the reason if not the height. For height, above a certain level, say 6 ft 2 inch, it starts becoming a liability in movement. Tennis used to…
I used to wear cotton long pants (thermals) all year, under the suit pant. Not letting the suit pant touch the skin made it easier to wear it for a very long time (multiple months) before dry-cleaning. Don't miss those…
I didn't mean to exaggerate. I think part of the improvement was the luxury of refactoring which should generally reduce the bloat. And, as someone else said, part of the issue is C++/Java, not static typing. When I…
A ~100k LOC project in a statically typed system with type hierarchies, interfaces, contracts and boiler plate will boil down to ~10k LOC in a dynamically typed language like Python. A 10k LOC project will be more…
Another advice is to have a thick skin and ignore the jerks, like for instance someone downvoting your note that was written with a good intention, sharing something learnt the hard way.
Can you please elaborate why? I worked in Manhattan for many years, lived a few as well, and moved out in 2009.
I think it's helpful to keep in mind that Python is general purpose and used in many domains in addition to data analysis (no matter which side of the walrus operator you are on)
A little over a year ago, there was an incident with a lot of sites being down and Cloudflare/Lumen being involved (per Hacker News chatter, I only ready about the incident here). I bought a few shares of each, at…
In the end, it's not about an OPEN SOURCE tool being perfect but whether it is helping you solving a problem. If it did not help you and YOU don't consider it production ready, then that's fine. But you seem to argue…
Do you have any citation on why "Dask doesn't consider their distributed version" to be ready? If it is your own view, then that's ok. I think dask is in heavy usage in real production systems. Let me cite one such…
We use dask heavily, along with rest of the pydata ecosystem. I guess we are in the 'sweet spot' where data doesn't fit memory, to begin with, but once we perform any filtering and aggregations, switch over to pandas.…
Not easy, unless you have an ability to identify opportunities that others don't have (such as applying your domain knowledge on a new trend, as I mentioned). You may do better than the market (market == passive index…
Technically, beating the market should be like being in the top 50 percentile of your class. The reason most big funds fail to do this is because they need to overcome the fees they charge, to match the market. If you…
I believe compression is still a work in progress in duckdb. My current use is with parquet files which compress well but don't lend themselves to updates like a db.
I work heavily with pandas and dask (when you want to use multiple cores), using parquet files for storage. We see a lot of benefits in selectively bringing in duckdb into the mix. For instance, the joins are extremely…
Many enterprises are coming up with patterns where they replicate the data from the database (say Redshift) into parquet files (data lake?) and directing more traffic including analytical workloads onto the parquet…
If you can completely stay away from Python/pandas, get all your work done with typed languages like Scala/Java, that's good. A lot of scientists and non-CS folks are using Python/R. They need to avoid mish mash of…
A little python training in the python stack (pandas/numpy/matplotlib or other visualization libraries) can go a long way to simplifying tech stack and get rid of these mind numbing BI tools. And companies are trying.…
Banks also have a huge cost of supporting legacy systems in Cobol, SAS.. even some 'modern' systems in Java/Python are a huge mess. https://www.efinancialcareers.com/news/2021/02/jpmorgan-stil... They mostly talk about…
< I hate finding CSVs that other data scientists Ideally you should be using the parquet format which will use the binary format, preserve column types and indexes [df.to_parquet(<file>); df = pd.read_parquet(<file>)]…
Mobil + Chevron => Moron
Growing up in a somewhat remote part of India, I would tune to BBC, Radio Australia to listen to test cricket commentary, on short wave. I have fond memories and owe a lot of my personal growth to SW.
Thanks for the clarification on this particular point. I was not aware of it.
I don't know all the history, but Magnus comes off as above the game. “I am playing at least one more day here in New York and, if I do well, another day after that,” Really? And I saw another quote that said "f*k you"…
Of late, I have been googling a lot of SAS and may have contributed to its rise in ranking! Not coding in SAS but moving SAS to Python. Speaking of enterprise Java, there's a ton of enterprise SAS and it's moving to…
Not sure why you were downvoted. Fitness, equipment (racket and strings) are the reason if not the height. For height, above a certain level, say 6 ft 2 inch, it starts becoming a liability in movement. Tennis used to…
I used to wear cotton long pants (thermals) all year, under the suit pant. Not letting the suit pant touch the skin made it easier to wear it for a very long time (multiple months) before dry-cleaning. Don't miss those…
I didn't mean to exaggerate. I think part of the improvement was the luxury of refactoring which should generally reduce the bloat. And, as someone else said, part of the issue is C++/Java, not static typing. When I…
A ~100k LOC project in a statically typed system with type hierarchies, interfaces, contracts and boiler plate will boil down to ~10k LOC in a dynamically typed language like Python. A 10k LOC project will be more…
Another advice is to have a thick skin and ignore the jerks, like for instance someone downvoting your note that was written with a good intention, sharing something learnt the hard way.
Can you please elaborate why? I worked in Manhattan for many years, lived a few as well, and moved out in 2009.
I think it's helpful to keep in mind that Python is general purpose and used in many domains in addition to data analysis (no matter which side of the walrus operator you are on)
A little over a year ago, there was an incident with a lot of sites being down and Cloudflare/Lumen being involved (per Hacker News chatter, I only ready about the incident here). I bought a few shares of each, at…
In the end, it's not about an OPEN SOURCE tool being perfect but whether it is helping you solving a problem. If it did not help you and YOU don't consider it production ready, then that's fine. But you seem to argue…
Do you have any citation on why "Dask doesn't consider their distributed version" to be ready? If it is your own view, then that's ok. I think dask is in heavy usage in real production systems. Let me cite one such…
We use dask heavily, along with rest of the pydata ecosystem. I guess we are in the 'sweet spot' where data doesn't fit memory, to begin with, but once we perform any filtering and aggregations, switch over to pandas.…
Not easy, unless you have an ability to identify opportunities that others don't have (such as applying your domain knowledge on a new trend, as I mentioned). You may do better than the market (market == passive index…
Technically, beating the market should be like being in the top 50 percentile of your class. The reason most big funds fail to do this is because they need to overcome the fees they charge, to match the market. If you…
I believe compression is still a work in progress in duckdb. My current use is with parquet files which compress well but don't lend themselves to updates like a db.
I work heavily with pandas and dask (when you want to use multiple cores), using parquet files for storage. We see a lot of benefits in selectively bringing in duckdb into the mix. For instance, the joins are extremely…
Many enterprises are coming up with patterns where they replicate the data from the database (say Redshift) into parquet files (data lake?) and directing more traffic including analytical workloads onto the parquet…
If you can completely stay away from Python/pandas, get all your work done with typed languages like Scala/Java, that's good. A lot of scientists and non-CS folks are using Python/R. They need to avoid mish mash of…
A little python training in the python stack (pandas/numpy/matplotlib or other visualization libraries) can go a long way to simplifying tech stack and get rid of these mind numbing BI tools. And companies are trying.…
Banks also have a huge cost of supporting legacy systems in Cobol, SAS.. even some 'modern' systems in Java/Python are a huge mess. https://www.efinancialcareers.com/news/2021/02/jpmorgan-stil... They mostly talk about…
< I hate finding CSVs that other data scientists Ideally you should be using the parquet format which will use the binary format, preserve column types and indexes [df.to_parquet(<file>); df = pd.read_parquet(<file>)]…
Mobil + Chevron => Moron