Generative AI workflows and hacks 2024

March 23, 2024 — October 4, 2024

economics

faster pussycat

innovation

language

machine learning

neural nets

NLP

stringology

technology

I will try to attempt synthesis of LLM research elsewhere. This where I keep ephemeral notes and links.

1 Chat API GUIs

Want a decent UX while using your own (possibly better-secured) API endpoint? Here are some options: NB, they all claim to keep your API key secret, but you should verify this yourself. Stealing API keys, or even better API keys plus confidential LLM queries, is a high-value target for attackers.

LibreChat

Enhanced ChatGPT Clone: Features Anthropic, AWS, OpenAI, Assistants API, Azure, Groq, o1, GPT-4o, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. Actively in public development.

dependency-heavy, so it prefers to be installed by Docker.
- danny-avila/LibreChat
- quick start
- weird overrides are necessary to get access to the custom endpoints which was my whole point for being here. Make sure that the librechat.yaml config is active. Once it is, configuration is flexible; e.g.
huggingface/chat-ui: Open source codebase powering the HuggingChat app

Seems to be more oriented to local LLMs, but handles remote ones too, e.g.
- 🤗 Chat UI
Open WebUI (Formerly Ollama WebUI)
- Open WebUI
- open-webui/open-webui
light install, but sensitive python version requirements. I had to use pyenv to get it to work.
```
pipx install --python (pyenv which python3.11) git+https://github.com/open-webui/open-webui.git
```
It does not seem to support multiple OpenAI Azure endpoints, which means being stuck to a specific model.
ChatGPTNextWeb/ChatGPT-Next-Web (Free. This one seems to work well) If you trust the maintainers, you can use the install deployed on the web.

Lovely UX. Error messages for custom endpoints are not great.
lobehub/lobe-chat: Much like chatgpt web, lovely UX. Also not great at endpoint switching.
MindMac (expensive native macos app)
ai-genie/chatgpt-vscode: Your best AI pair programmer in VS Code A functional client. If all you care about is code generation, this is a good choice. If you want other features, such as flexibility, easy switching of endpoints etc, no good.

The next two I have heard recommended but have not tested:

2 Multi-agent LLMS

Autogen

3 Misc

Calculating Sunflower Oil Production (ChatGPT psychosis)
Welcome to LangChain — 🦜🔗

Large language models (LLMs) are emerging as a transformative technology, enabling developers to build applications that they previously could not. But using these LLMs in isolation is often not enough to create a truly powerful app - the real power comes when you are able to combine them with other sources of computation or knowledge.

This library is aimed at assisting in the development of those types of applications.
Prompt Airlines - AI CTF by Wiz

Your goal is to manipulate the customer service AI chatbot to get a free airline ticket

4 Running locally

First, get a fancy fast computer. Then: