pull down to refresh
169 sats \ 3 replies \ @SimpleStacker 12 Dec \ on: Stacker Saloon
One annoying thing about research work: how you build a query can have orders of magnitudes differences in how long it takes to complete the query. But I don't really know if it's worth optimizing a query that I really only need to do one (or a few) times. On the other hand, I don't like waiting 10 minutes for my query to complete.
reply
There's a bit of uncertainty there. If I do everything correctly the first time, I only need to run it once. But if I make mistakes, or realize I need other data fields / other calculations, I may need to run it again. Right now my inefficient version takes about 18 minutes to run. If I have to re-do it 3 or 4 times, it gets into the hours.
As for whether to optimize, there's uncertainty about the gains I'd be able to achieve and how long it'd take me to figure out a better approach. I suspect there is though, based on known structures in the data.
And then, here I am, wasting time thinking and talking about it.
I probalby won't optimize though, because those 20 minute runs can be filled with other things, like posting on Stacker News.
reply
reply