JetBrains, the corporate behind a variety of standard app improvement instruments, has launched its first “open” AI mannequin for coding.
On Wednesday, JetBrains made Mellum, a code-generating mannequin the corporate launched for its varied software program improvement suites final yr, brazenly out there on the AI dev platform Hugging Face. Mellum, skilled on greater than 4 trillion tokens, weighs in at 4 billion parameters, and is designed particularly for code completion (i.e. finishing code snippets primarily based on the encircling context).
Parameters roughly correspond to a mannequin’s problem-solving abilities, whereas tokens are the uncooked bits of information {that a} mannequin processes. 1,000,000 tokens is equal to ~30,000 traces of code.
“Designed for integration into skilled developer tooling (e.g. clever code options in built-in developer environments), AI-powered coding assistants, and analysis on code understanding and technology, Mellum can be well-suited for instructional functions and fine-tuning experiments,” explains JetBrains in a technical report.
JetBrains says that it skilled Mellum, which is Apache 2.0-licensed, on a set of information units together with permissively licensed code from GitHub and English-language Wikipedia articles. Coaching took round 20 days on a cluster of 256 H200 Nvidia GPUs.
Mellum takes some work to stand up and operating. The bottom mannequin can’t be used out of the field; it needs to be fine-tuned first. Whereas JetBrians has supplied a number of Mellum fashions fine-tuned for Python, the corporate cautions that they’re meant for “estimation about potential capabilities” — not deploying right into a manufacturing atmosphere.
AI-generated code is little question altering how software program is constructed, however it’s additionally introducing new safety challenges. Greater than 50% of organizations encounter safety points with AI-produced code typically or often, in accordance with a late 2023 survey by developer safety platform Synk.
Techcrunch occasion
Berkeley, CA
|
June 5
Certainly, JetBrains notes that Mellum might “replicate biases current in public codebases” (e.g. producing code related in type to open supply repositories), and that its code options received’t essentially be “safe or freed from vulnerabilities.”
“That is only the start,” JetBrains wrote in a weblog put up. “We’re not chasing generality — we’re constructing focus. If Mellum sparks even one significant experiment, contribution, or collaboration, we’d take into account it a win.”
