Kx released a 32bit, free-for-commercial-use version in 2014 and then reversed course around a year later after banks and hedge funds surprised them by flocking to it for a large subset of their developers / dev machines.
Hopefully they stick to it this time around. It's an incredible system. It's the only thing I've ever used, including Pandas, dplyr & Matlab, where someone could stand over my shoulder asking data analysis questions and I could answer them, on the fly.
LLM's, though notoriously bad (so far) at KDB+/Q compared to other languages, are still a godsend for folks getting started. I recently returned to writing Q after being away from it for 8 years and I've been amazed how good even Google's AI suggestions have been at helping with functions & queries.
Getting started tip: try using Q strictly as a query language, avoid K. Do everything else (data shoveling, devops,...) with a different language.
Could you give an example of where it shines? I have had to answer many a SQL/pandas question with someone over my shoulder, so curious where you see the sharpest benefit.
Admittedly, I am unlikely to learn this proprietary DSL, but always good to know what is the best tool for a job.
Q is not just a query language or a database. It's an array programming language with database capability. You can build an entire framework with just Q, from real-time streaming, to in memory database to on disk database. You can build all APIs and business logic around it. Because it's vector oriented and in memory it's faster than pretty much everything else, no loops required. I have seen a team of 15 KDB developers build what would require an entire etrading department of 200+ developers
I've been surprised to see many pro devs using Excel/VBA as a secondary tool.
One example: a couple years ago I was working with a big hedge fund and one of their data analysts sent me an Excel model he had built and I was tickled to see the .xlsm extension (i.e., VBA code on board).
"Ahh ha", I thought, "Let's see what these macro-recording cowboys have been up to."
There was a lot of VBA inside, all written by this Caltech comp sci data analyst who was a Python superstar. The VBA was for pulling data from a database, putting it on a sheet, building some formulas, and some pretty formatting. There were even a few userforms!
I teased him, "VBA? What else are you guys using over there? A cotton gin and a steam shovel?"
I was startled to hear him heap praise upon Excel and VBA instead of the usual complaints.
He said something that stuck with me, "Excel makes it easy to understand the dependency structure that is implied by computations. If I had done this in Python, I'd be answering questions about it all day long."
As I have gained experience as a developer using the right tool for the right job becomes paramount. And the lazy answer can be much better than some incomprehensible mess of ideas.
VB6 has a pretty big community, and https://twinbasic.com/ has really helped unify VBA and VB6 communities as of late. So it might have a little of a resergence in the dev community.
I'm always stunned to see the level of VB6 expertise that remains in the world even two decades after Microsoft left it for dead.
Just look at the effort and knowhow that went into this VBA function that resolves the local file system path from the https url of workbooks synced to OneDrive/SharePoint:
A business I run depends heavily on Google Sheets that I inject values into and read computed values out of. That lets us define rather complex business logic in spreadsheet form, and business and finance people can easily tweak it. Everybody is very happy with this solution.
We started homeschooling by happenstance when it turned out that kindergarten was going to conflict with our son's beloved jiu-jitsu class. The class was a bit of a drive and we weren't going to be able to get there in time for his favorite part: the 30 minutes of horsing around on the mats before class started.
We thought, "Well, it's only kindergarten". It worked out well enough (knock on wood) that we just kept going.
Compared to my own harried "THE BUS IS COMING!!!" sleep-deprived school days, it has been a wonderful change.
If I could go back in time, I'd put my mom in an ankle lock until she agreed to do the same for me.
Especially since I learned that in my home state homeschoolers can participate in high-school sports. In high school I had to wake up at 4AM for 5AM hockey practice. If I could have returned home afterwards and slept, rather than going straight to school, I would have thanked the Gods.
May I ask if your son is an only child? I’ve thought about online homeschooling my son when he’s old enough (he’s still a baby), but I wonder if he would be lonely if he turns out to be our one-and-done. You mentioned your son’s beloved jiu-jitsu; does he have other outlets for meeting friends and socializing?
He has a younger sister. As for other outlets for making friends and socializing, team sports are indispensable. Also great, and surprisingly underutilized -- in my observation -- are playgrounds.
With playgrounds you get the best part of school, recess, without the rest of it.
If parents would look up from their phones for a minute and watch what's going on in playgrounds they'd be amazed.
Recently I saw about 20 kids, across a wide age range, hold a spontaneous, well-organized game of Among Us.
You won't just make friends there; you'll run in Draco Malfoys too; and that's also valuable.
Most only-childs wish they had siblings, and many one-of-manys wish they were only childs. My wife was raised as an only child and is stunned by how much our children fight with each other.
On the other hand, many of the only children I know (I'm close friends with 4, all from different backgrounds - one a non-US childhood) are all quite creative and independent people. Sometimes being allowed to be bored is excellent.
I agree that insider trading is an unlikely explanation for the reason you mention.
It could be that an early algorithmic advantage compounded into an incumbency advantage: better research tools; better data; better pay for better scientists.
The 'secret sauce' may have started out as novel discoveries and, over time, become 'just being RenTec'.
I have been consulting at a world-renown hedge fund that has a lot of F# including nightly processes that have apparently been running for years with zero maintenance. You'll never hear them talk about it or see them at meet-ups though. The CTO is a bit of a maverick -- I'm not suggesting that this type of quiet adoption is common.
Sadly in my experience it is common for F# given its finance background at least historically, particularly in London/Europe. There is a culture of not showing your cards in these environments having worked in a few of them, and quietly being a success. Its an "arms race" with your competitors after all - only people in the industry jumping between companies have an idea what their other competitors are using often sometimes with "non-compete" clauses. Meetup's/conferences/articles showing your work are often explicitly banned or at best not encouraged by the companies for commercial reasons.
If you use collection classes in vba you can run into this bug pretty quickly. Last year I deployed an Excel add-in to a client that enabled them to run Snowflake queries concurrently from vba and this line:
If queries.FilterHasError().Count > 0 Then
was being evaluated as true even when the count was zero.
I had developed the add-in using 32-bit Excel. This was an extremely confounding troubleshooting experience. Eventually Google & Stackoverflow rescued me from believing I was going crazy.
This debate about whether to use named-ranges in Excel formulas brings back some distant memories. I was on a team at Goldman Sachs that built Excel tools for the investment bankers and because of that we'd help out with the new banker Excel training held at the giant Chelsea Piers sports complex.
Every year freshly minted MBA's would begin the training and immediately become aghast when the trainers told them to never, ever, use named-ranges in formulas. Not only had the trainees been explicitly taught to use named-ranged in their MBA programs, but any idiot could plainly see that [=enterprise_value/ebitda] was better in every way than [=C13/F22]. More expressive, more readable, easier to spot errors, etc.
The trainers would argue that in an MBA program you build your models, submit them, and move on to the next assignment, so you don't get a view of the longer-term problems that arise from named-ranges. What does [=ebitda] actually tell you? Is that the last quarter's actual number, is it the current estimate of the next fiscal quarter, is it a 12-month blended forward estimate?
And as you try to solve those questions with [=ebitda_est_next_fiscal_qtr] you often end up with two more problems: an unambiguous name for you can unambiguously mean something else to someone else and as you update your model over time, if you forget to update your named-range references, you have formulas that look right, but are wrong, e.g., [=ebitda_est_next_fiscal_qtr] now refers to an actual reported ebitda number, not next quarter's estimate.
[=C47] while not telling you much at all, is unambiguous. It doesn't look "right" or "wrong". It can't mislead you. If you want to know what it is, go look at cell C47. The new MBA's would argue, "Wait a second, C47 could now point to the wrong data as well, and it could be mislabeled with a stale row or column header. And the trainers would reply, "Exactly, and when you go to see what C47 represents you will have the context to recognize those errors and fix them."
heh, your exchange looked pretty much like a holywar between programmers. Except problems looked like if you're discussing merits of different old varieties of BASIC, with GOTO and line numbers. I really don't see how excel is more visual, or easier to grasp after adding just a several columns of data or formulas.
I wonder if there could be made some middleground language, visual enough to not being scary for spreadsheet people, and debuggable and readable enough to not become a mess when model becomes big. Jupyter comes to mind, but it looks like it didn't get any traction outside data science.
I encountered exactly this when my brother went to film school. The students would rotate roles and it was his turn to be the producer. One day I came home from work and he was having problems with his Excel-based budget sheet. I noticed that his row of sums at the bottom of one sheet had hard-coded values rather than formulas. Thinking he had inadvertently pasted over them, I told him, "Look, you killed your formulas." "What are formulas?", he asked. Turns out that he had been tallying up the columns with a calculator and then typing those totals into Excel.
Same for me. My high school and college hockey coaches could really let you have it. They never pulled you aside and did it in private either. My college coach had episodes that would make even Bobby Knight look like a pussycat. He once had a roll-on-the-floor grappling fight with a teammate in the locker room between periods. (Coach had a big tactical advantage: he wasn't wearing skates) When I ended up working on a trading desk the impromptu performance review broadsides -- in front of everyone -- felt very familiar.
More color on the original FoxPro matching engine:
"Levine found a simple but hugely consequential 'trick' to speed matching. In Levine’s later paraphrase, what the 'enter2order' procedure did when the Island system received a new order was to: 'See if there was a record from a recently cancelled order that we can reuse for this new order. This is hugely important because that record will likely still be in the cache [fast internal memory] and using it will be much faster than making a new one. (Levine, n.d.)'"
Hopefully they stick to it this time around. It's an incredible system. It's the only thing I've ever used, including Pandas, dplyr & Matlab, where someone could stand over my shoulder asking data analysis questions and I could answer them, on the fly.
LLM's, though notoriously bad (so far) at KDB+/Q compared to other languages, are still a godsend for folks getting started. I recently returned to writing Q after being away from it for 8 years and I've been amazed how good even Google's AI suggestions have been at helping with functions & queries.
Getting started tip: try using Q strictly as a query language, avoid K. Do everything else (data shoveling, devops,...) with a different language.