2024
April – June
-
I have a suspicion that using LLMs for generating code is going to boost the functional core, imperative shell paradigm. In the short term.
-
In a noisy ecosystem, curation of signal is important.
-
"Sincere eagerness to rigorously engage and reorient computer culture is a must." Intriguing. https://chrisnovello.com/teaching/risd/computer-utopias/
-
Really appreciate the work from @winglian and @axolotlai. Axolotl is the tool driving and abstracting away all the tedious details of fine-tuning in my recent posts.
-
Ok, now I have the @weightsbiases integration working with the @modallabs training of @MistralAI. It's nice to have a pleasant dashboard for your work.
-
The videos from bbuzz are already online: https://www.youtube.com/playlist?list=PLq-odUc2x7i8jHpa6PHGzmxfAPEz-c-on
-
Here's what llm fine-tuning looks like, using @modallabs (cool service!) This is a snapshot of my first ever llm fine-tuning, in progress today. It's fine-tuning a Mistral-7B model, using the materials in their example docs and repo. Thanks to @charlesirl for the credits and to…
-
The topic of the month for June is "systematic". The topic of the month for May was "vibe check".
-
Cool stuff. I really need to spend some time with Instructor. https://useinstructor.com/blog/2024/06/06/enhancing-rag-with-time-filters-using-instructor/
-
Sorry. I know I can do better. https://shiptalkers.dev/compare?github=codycollier&twitter=cmcollier
-
Upgraded @vespaengine again this morning. It went smoothly. Again. There's so much experience baked into this product.
-
Are auto-regressive LLMs deterministic? Yes in theory, which was my intuition and understanding. It's less true in practice though, and Perplexity summarized this very well: https://www.perplexity.ai/search/Theoretically-is-a-qr0ecAieSrKlXcCiWGzGwQ I've seen this come up more th…
-
It's startling to see how different people define RAG. See the HN comments on this story: https://news.ycombinator.com/item?id=40431210
-
Key takeaway today: "vibe check" is the official & formal first eval step when fine tuning an LLM 🫡 https://x.com/cmcollier/status/1793004587009348080
-
This is going to be fun... https://maven.com/parlance-labs/fine-tuning
-
My playground Search and RAG over all 40M hacker news items. Built on top of @vespaengine and the HN api. https://orangewords.com/
-
How are Embeddings Affecting Traditional Text Search? In this post, I provide an informal explanation of lexical, semantic, and hybrid search for text documents. It's targeted at non-technical readers in the industry. Enjoy! https://codycollier.com/tx/search/lexical-semantic-and…
-
Take deliberate actions
-
There's so much going on in tech and AI. We're getting to watch a slow motion collision between multiple domains over the last few years. There's: traditional information retrieval learned representations and embeddings modern language models (bert and friends) the large languag…
-
I keep having to re-check my calculations. Can I really fit decent quality embeddings for my 40 million docs into a few GB of ram?! Vespa integrating MRL + binarization + bit packing for a huge win 🔥🔥 Looking forward to working through this post and trying it out on my data.……
-
This is a nice set of lessons, from Ken @ http://gettruss.io. It covers basic usage of the OpenAI Chat Completions API. Some of them mirror my own experiences building basic RAG systems over the last few months. Lessons after a half-billion GPT tokens https://kenkantzer.com/less…
-
Making even small bits of progress each day, is like the compound interest of building and shipping products. When shipping is your priority, you can more clearly see things that can wait until later.
-
I think a dream job might be working the fittings, fasteners, and fixtures aisles of hardware stores. Spending time getting everything in the right place and helping people find just the right solution all sounds relaxing.
-
I'm working on something new today. Starting is hard. I was reminded yet again how many ways my brain can find to procrastinate. It's procrastivity. One more this or that. One more look at some library or blog post. Opening a terminal and allowing v0 to be a hacky mess of code r…
-
Forget about building MVPs. Create an MVX. What is the minimum valuable experience you can provide to a customer?
-
This year's Haystack Search Relevance conference has a nice lineup of talks. Looking forward to it! I definitely recommend picking up a virtual attendance ticket if you're not going in person (only $40). https://haystackconf.com/2024/
-
Totality is unreal