The conversation

Lessons learnedExperience excerpt · English

I used to work for a Go shop

I used to work for a Go shop. We dealt with financial data. I found it so annoying that many of my colleagues would use Go for one-off tasks such as aggregating CSV files, updating the database with some data, or fetching data from the database, and then trying to make a plot. I saw my colleagues again and again implementing basic algorithms such as rolling median, or finding a maximum. Instead of loading data into Pandas and doing a group by, they would create some kind of loopy solution that would use maps. I totally understand why some people prefer Go in production to Python, but I could never understand why people wouldn't just learn the standard data science tools instead of reinventing the wheel in Go, always debugging their own off-by-one errors. It was difficult for me to trust the results of such analyses, given that I knew how many of the basic functions were written on the fly and probably not even tested. In the end, I didn't think it was a good use of the company's time. It felt more like an ego thing - thinking and showing that Go is sufficient. …

Original discussionGo is my hammer, and everything is a nail798 source comments · at import
8

Community replies 0

No community replies yet. Add your perspective.

Replies from the source 8

Direct replies available in this import. Original-language excerpts.

PH

I've had exactly the same experience, it's a nice language but using it for things it's not suited for like data exploration makes no sense to me. Production data pipelines on the other hand, but only after testing them well and as you say, making sure there's good testing if you're implementing things like numerical routines.

Full original
AR

You don’t need to spend time learning standard data science tools. These tasks are well-defined, thoroughly documented in different blogs, and with today’s AI-driven code generation, basic programming knowledge is sufficient. Attempting to manipulate CSV data and generate visualizations in Go would be a waste of time, delivering subpar results.

Full original
TH

> I used to work for a Go shop. We dealt with financial data. People dealing with finance are THE MOST risk-averse people I know. No new tool will be used without years of vetting and it'll still be blamed if something goes wrong - even if the new tool never touched that bit of the process =) Source: Consulted for a finance company and it was Ye Olde Java and COBOL all the way down =)

Full original
CL

> Instead of loading data into Pandas and doing a group by SQL can do `group by`. No need for overkill with "data science tools".

Full original
WE

I have done the same like your colleagues. Recently I discovered that pandas + jupyter notebook is much better tools. However as gopher that needs occasional data cruncher, I totally relying on LLM chat agent to do my work. But the rest of the work is still go, so the context switching is there but bare able. Imagine a go crawler dumping csv and a python script crunching the csv.

Full original
UM

I've mostly seen the opposite - pandas and jupyter notebooks shipped directly to production because the data scientists and AI guys didn't know how to do anything but python. As a result, the solutions were not performant and often had lots of runtime crashes due to python's more loose typing

Full original
WH

> I used to work for a Go shop. We dealt with financial data. Hopefully you never forgot to initialize a numeric field and had it default to 0!

Full original
RA

> rolling median, or finding a maximum The real question is why are you writing code to make a plot when you can load a csv in excel and get a clean plot in 2 minutes.

Full original
Continue on Hacker News

Respect different perspectives. Share concrete experiences.

Sign in to post, reply, repost, and quote.