Generative AI workflows and hacks 2024
March 23, 2024 — December 12, 2024
I will try to synthesise LLM research elsewhere. This is where I keep ephemeral notes and links.
1 Standards
-
Large language models (LLMs) are emerging as a transformative technology, enabling developers to build applications that they previously could not. But using these LLMs in isolation is often not enough to create a truly powerful app — the real power comes when you are able to combine them with other sources of computation or knowledge.
This library is aimed at assisting in the development of those types of applications.
2 Chat API GUIs
Want a decent UX while using your own (possibly better-secured) API endpoint? Here are some options: NB, they all claim to keep your API key secret, but you should verify this yourself. Stealing API keys, or even better API keys plus confidential LLM queries, is a high-value target for attackers.
Some are listed in reorx/awesome-chatgpt-api, for web and desktop.
My experience thus far is that they are all terrible for my purposes. I would personally like to point a chat client at an API endpoint. I would like to switch endpoints occasionally (e.g. because I want to use different models). I would like to have a nice UX. I don’t need to share anything, I don’t want to require a horrible install process. I would like error messages to be clear and simple. Optional: I would like to persist chats.
This feature combination is unavailable. Read on for various options.
LibreChat
Enhanced ChatGPT Clone: Features Anthropic, AWS, OpenAI, Assistants API, Azure, Groq, o1, GPT-4o, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. Actively in public development.
This claims to have all the features I want, except for an easy install process. In fact, its install process is horribly hard, and after 3 hours of trying I have not been able to get it to work.
Dependency-heavy, so it prefers to be installed by Docker. Per default it is an all-singing, all-dancing, multi-user multi-backend chat client, with persistent storage and a lot of features. Way too many features. If I want to talk to an LLM I do not need hundreds of megabytes of dependencies wrapping up into a docker image that makes all the error messages baffling, and then ultimately doesn’t do that one thing I wanted.
- danny-avila/LibreChat
- quick start
- weird overrides are necessary to get access to the custom endpoints which was my whole point for being here. Make sure that the
librechat.yamlconfig is active. Once it is, configuration is flexible; e.g.
The other (?) Librechat? Is that the same? I dunno it cannot be more ponderous, surely?
huggingface/chat-ui: Open source codebase powering the HuggingChat app
Seems to be more oriented to local LLMs, but handles remote ones too, e.g.
Open WebUI (Formerly Ollama WebUI)
Light install, but sensitive Python version requirements. I had to use pyenv to get it to work.
It does not seem to support multiple OpenAI Azure endpoints, which means being stuck to a specific model.
anything-llm
Privado-Inc/MuroChat: MuroChat open-source LLM chat application Installation Guide Does not seem very active.
ChatGPTNextWeb/ChatGPT-Next-Web
Lovely UX. Error messages for custom endpoints are not great. If you trust the maintainers, you can use the install deployed on the web.
lobehub/lobe-chat: Much like ChatGPT web UI, lovely UX. Also not great at endpoint switching.
MindMac (expensive native macOS app)
ai-genie/chatgpt-vscode: Your best AI pair programmer in VS Code A functional client built into an IDE. If all you care about is code generation, this is a good choice. If you want other features, such as flexibility, easy switching of endpoints etc, no good.
The next two I have heard recommended but have not tested:
3 Multi-agent LLMS
4 Surveys
- RUCAIBox/LLMSurvey: The official GitHub page for the survey paper A Survey of Large Language Models. / [2303.18223] A Survey of Large Language Models
6 Misc
stackblitz/bolt.new: Prompt, run, edit, and deploy full-stack web applications
Calculating Sunflower Oil Production (ChatGPT psychosis)
Prompt Airlines - AI CTF by Wiz
Your goal is to manipulate the customer service AI chatbot to get a free airline ticket
7 Running locally
First, get a fancy fast computer. Then:
5 Social history