Code generation, programming assistants

Turing-complete autocorrect, vibe-coding, …

2021-10-14 — 2025-04-30

faster pussycat
language
machine learning
making things
neural nets
NLP
signal processing
stringology
UI

Suspiciously similar content

A cousin to neural automata: writing machines to code for us, because code generation is fancy text generation, which involves similar technology, i.e. large language models.

There are two aspects to making this work: the model and the interface. Sometimes the two are combined, as in Cursor, which makes it hard to structure this page.

Figure 1

1 Security

So, the obvious important thing first. I am vaguely concerned about how much of the world uploads their source code to these code servers. The potential for abuse is huge.

Anyway, the arms race is real, so let’s all ignore that and upload all our code into their models, eh?

2 MCP

  • MCP — Model Context Protocol

    MCP is an open protocol that standardises how applications provide context to LLMs. Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standard way to connect your devices to various peripherals and accessories, MCP provides a standard way to connect AI models to different data sources and tools.

It’s one of those things that’s very easy to intuit if you use it, but hard to explain.

Notable clients I’ve used include the Cursor built-in and the Claude-Desktop MCP client.

Nifty MCP servers

3 Clients

Code processing is just text processing, when you think about it, so the AI text tools are also useful here.

3.1 Claude-code

anthropics/claude-code: “Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflows — all through natural language commands.”

It is a command-line but a really nicely designed one that interacts gracefully with a normal IDE. I currently prefer it to GitHub Copilot for most implementation tasks.

Docs here.

It works better if we install ripgrep.

Claude is very eager and requires some skills to use properly. Be stingy in what you permit it to do, and keep extensive design docs to keep it on-scope. Remind it constantly to check the docs.

The main default config dir for claude-code is ~/.claude/. (It also rudely stores stuff in ~/.claude.json.) Multiple configs are supported via multiple config dirs, which are supplied in env vars:

CLAUDE_CONFIG_DIR=~/claude_work claude

3.2 Aider

From Harper Reed’s Blog post:

Aider - AI Pair Programming in Your Terminal.

It seems more sleek and minimalist than claude-code. The voice programming mode is cute.

IDE mode:

If you run aider with --watch-files, it will watch all files in your repo and look for any AI coding instructions you add using your favourite IDE or text editor.

Comments like these

# Make a snake game. AI!
# What is the purpose of this method AI?

3.3 GitHub Copilot

GitHub Copilot uses suggestions from OpenAI Codex some GPT-4 off-the-shelf thing now (IIRC?) to suggest code completions. The original Codex engine was really good and I don’t think the general-purpose models have matched it, years later.

Figure 2: Looks like AI Safety is going fine in GitHub Copilot.

GitHub Copilot has a great workflow for automatic completions, and it was that that originally led me to pay them money. Since then they have rolled out extra chat interfaces. The new generation of tools are janky and semi-reliable. It is bad at following instructions, messes up basic stuff like indentation and is not especially fast. Occasionally it will forget that it is editing code and simply talk about editing code, or dump weird repeated sections of code into the code, or just delete random stuff. It’s a little bit like coding with a drunk genius. Still a net positive, but not great; They squandered their early lead.

Behind a firewall, we need at least the following whitelist exceptions:

  • vscode-auth.github.com
  • api.github.com
  • copilot-proxy.githubusercontent.com

See Networked VS Code for more whitelist rules we need for VS Code.

3.4 Cursor

Cursor - The AI-first Code Editor

Cursor is an AI-powered code editor that helps you build software faster.

It is a VS Code fork with its own AI engine (“Copilot++”, cheeky) and some extra UI affordances. My colleagues assure me that this leads to way fewer annoying psychoses and sidetracks than Copilot.

3.5 Fauxpilot

FauxPilot: Like GitHub Copilot without Microsoft telemetry:

Updated GitHub Copilot, one of several recent tools for generating programming code suggestions with the help of AI models, remains problematic for some users due to licensing concerns and the telemetry the software sends back to the Microsoft-owned company.

fauxpilot/fauxpilot: FauxPilot - an open-source alternative to GitHub Copilot server

This is an attempt to build a locally hosted alternative to GitHub Copilot. It uses the SalesForce CodeGen models inside NVIDIA’s Triton Inference Server with the FasterTransformer backend.

Being able to work offline would be a real win; Copilot loves bandwidth too much.

3.6 Continue

For JetBrains and VS Code IDEs, Continue is a plugin that provides AI-powered code completions. Seems to support BYO model. Have not tried.

3.7 Cody

Cody | AI coding assistant

Cody supports the most powerful LLMs including Claude 3.5, GPT-4o, Gemini 1.5, and Mixtral-8x7B.

You can also bring your own LLM key with Amazon Bedrock and Azure OpenAI.

4 Models and serving them

4.1 Codeium

Codeium

Codeium has been developed by the team at Exafunction to build on the industry-wide momentum on foundational models. We realised that the combination of recent advances in generative models and our world-class optimised deep learning serving software could provide users with top-quality AI-based products at the lowest possible costs (or ideally, for free!).

4.2 Codestral Mamba

Codestral Mamba | Mistral AI | Frontier AI in your hands

Following the publishing of the Mixtral family, Codestral Mamba is another step in our effort to study and provide new architectures. It is available for free use, modification, and distribution, and we hope it will open new perspectives in architecture research. Codestral Mamba was designed with help from Albert Gu and Tri Dao.

Unlike Transformer models, Mamba models offer linear time inference and can theoretically model sequences of any length. This efficiency is especially relevant for code productivity use cases — this is why we trained this model with advanced code and reasoning capabilities, enabling it to compete with state-of-the-art transformer-based models.

4.3 Ollama/LLaama coder

Two offline solutions that work well together:

Ollama

Run Llama 3.1, Phi 3, Mistral, Gemma 2, and other models. Customise and create your own.

Llama Coder

Llama Coder is a better and self-hosted GitHub Copilot replacement for VS Code. Llama Coder uses Ollama and CodeLlama to provide autocomplete that runs on your hardware. Works best with Mac M1/M2/M3 or with RTX 4090.

4.4 Amazon CodeWhisperer

AI Code Generator - Amazon CodeWhisperer - AWS

Available as part of the AWS Toolkit for Visual Studio (VS) Code and JetBrains, CodeWhisperer currently supports Python, Java, JavaScript, TypeScript, C#, Go, Rust, PHP, Ruby, Kotlin, C, C++, Shell scripting, SQL and Scala. In addition to VS Code and the JetBrains family of IDEs — including IntelliJ, PyCharm, GoLand, CLion, PhpStorm, RubyMine, Rider, WebStorm, and DataGrip — CodeWhisperer is also available for AWS Cloud9, AWS Lambda console, JupyterLab and Amazon SageMaker Studio.

Free for individual use.

4.5 Others

5 Pedagogy

Coding assistants are a great way to learn to code (if that is a thing that is still valuable to do?)

6 Data processing

yamadashy/repomix: 📦 Repomix (formerly Repopack) is a powerful tool that packs your entire repository into a single, AI-friendly file. for when you need to feed a codebase to LLMs.

repomix --remote https://github.com/yamadashy/repomix

simonw/files-to-prompt is another favourite (“Concatenate a directory full of files into a single prompt for use with LLMs”). See files-to-prompt for background.

7 Incoming

Figure 3

Querying Glean:

Glean is a system for working with facts about source code. It is designed for collecting and storing detailed information about code structure, and providing access to the data to power tools and experiences from online IDE features to offline code analysis.

For example, Glean could answer all the questions you’d expect your IDE to answer, accurately and efficiently on a large-scale codebase. Things like:

  • Where is the definition of this method?
  • Where are all the callers of this function?
  • Who inherits from this class?
  • What are all the declarations in this file?

8 References

Beurer-Kellner, Fischer, and Vechev. 2022. “Prompting Is Programming: A Query Language For Large Language Models.”
Bubeck, Chandrasekaran, Eldan, et al. 2023. “Sparks of Artificial General Intelligence: Early Experiments with GPT-4.”
Din, Karidi, Choshen, et al. 2023. “Jump to Conclusions: Short-Cutting Transformers With Linear Transformations.”
Suzgun, Scales, Schärli, et al. 2022. “Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them.”
Wang, Wei, Schuurmans, et al. 2023. “Self-Consistency Improves Chain of Thought Reasoning in Language Models.”