The original vulnerability description is not worded very well, here's my understanding of what's going on:
1. Attacker crafts a malicious Chart.yaml containing arbitrary code
2. Replaces Chart.lock with a symlink pointing to a sensitive file (like .bashrc or other startup scripts)
3. When you run helm dependency update, Helm processes the malicious Chart.yaml and writes the payload to whatever file the symlink targets
4. Code executes when the targeted file is next used (e.g., opening a new shell)
Why This Works:
Helm follows the symlink during the dependency update process without validating the target, allowing arbitrary file writes outside the intended chart directory.
Can anyone explain in what setup an attacker who can create a symlink where Chart.lock was could not directly write .bashrc or similar? Is this related to how Git handles symlinks?
This has nothing to do with Git. A symlink can be packaged up in a tarball and shipped from one system to another. An attacker would need to create a malicious Chart.yaml file and a Chart.lock file pointing to another file. Then ship those to a system where dependencies are then updated.
This doesn't affect things like installing or upgrading a chart. Dependencies aren't updated at that time.
A symlink is just a special file that contains a string of text, it's not tightly bound to the target like a hard link. You can write anything into that string of text, including, say, "~/.bashrc". Then you can ship that symlink onto another system, and it suddenly points to your .bashrc.
Git just moves symlinks across systems as is, so yes, you can use git to deploy the exploit.
Helm is a program that allows users to creates packages which other users consume. Those packages contain files that are normally generated by Helm itself, but apparently if you alter your package definition by hand you can replace Chart.lock with a symlink.
As I'm typing this it's occurring to me that you probably shouldn't be able to do that. The fix they applied was to prevent the actual write from occurring when trying to write the lockfile and determining that the lockfile is a symlink. They could (should?) also validate that like, the package itself hasn't been screwed with in this manner.
Having read the CVE multiple times I am still unsure how 2. above happens? Is it possible through the malicious chart itself or is it a dependency for the CVE to be in play at all? And if the latter - what local process would write a symlink from a helm lock file to any kind of system start up script which doesn't point to a much bigger problem than this CVE?
The attacker creates a symlink (e.g., using `ln -s`) to another file. The attacker needs to create the malicious Chart.yaml file and symlink that the Chart.lock file points to.
If being able to create files and symlinks to them is a pre-condition for this, then it's not a serious security bug. If you have that kind of access then there are a million nefarious things you can do.
This is almost becoming a joke at this point, "assuming an attacker has access to the system, they can change things on the system".
You place malicious binaries outside the helm directory. Helm will now execute malicious code through the helm chart pointing outside the helm directory.
Don't I have already bigger problems if you have access to my file system to place there malicious code?
Is the danger here that one can get an execute permission? But if you can manipulate my helm chart why can you not also place the malicious code in the helm directory?
Yeah, there is a rather strong "downloading and executing arbitrary code from the Internet may lead to execution of arbitrary code" kind of vibe there.
I think that there are "legitimate" use cases for symlinks that read from outside the root, which at this point are probably looked upon even less favorably. It's likely that making the change you're proposing would be backwards incompatible.
I agree that it's not clearly explained why this isn't a concern though. A cursory search for other instances of os.WriteFile doesn't seem to surface any thorough controls...
edit: ok actually it looks like the lockfile is special because it's the only instance of helm itself directly writing a file on behalf of a package consumer
If you have a chart that has `deploy.yaml` symlinked to `/home/john/testcharts/redis/deploy.yaml`, that chart is clearly not going to work on anyone's machine except john's, so that chart is useless on anyone else's machine.
If you're saying the use-case is for charts that aren't distributed, well, I'm saying we should ban all symlinks on distribution (downloading and unpacking a chart should fail if it has symlinks outside of the root), and I just can't imagine any use-case where a distributed chart with external symlinks makes sense.
If this whole thing is about charts that aren't distributed, but local to some developer's machine, well, in that case who cares if the developer can pwn themselves by typing "ln -s ~/.bashrc Chart.lock", they could have just pwned themselves by typing "bash" even more quickly.
I have use cases for linking Terraform lock files to keep various deployments/modules on consistent versions. I could see there being a use case for symlinking Chart.lock files although usually that's limited to an internal implementation and not something a general purpose chart would probably ship
i.e. you have 3 different charts that all depends on `cache`, `load balancer` and `database` charts and you want to only ever have 1 version deployed of those subcharts so you want the parent chart locks linked
Ya, I mean, I put "legitimate" in quotes for a reason. I think most people agree with you. This has been a thing that they've been aware of and struggling with for a while.
Smattering an --allow-symlinks flag all over their commands seems to be the least inelegant way to handle this while still giving users an easy way to maintain compatibility. Maybe they'll come around to it after this.
Helm is an abomination, as the whole idea of using a text template engine to generate YAML is. And this vulnerability adds insult to injury ;)
Sorry, just can't really recover from trauma of counting spaces and messing up newlines, etc. when writing Helm templates. You know, Lisp "sucks" because "you need to count parenthesis" (you actually don't), yet Helm is a widely accepted technology where you need to count spaces for (n)indent ;)
What is the attack scenario here? Where are the security boundaries? How does the attacker gets their repository with a symlink in it to the victim? Is Helm typically run as a privileged user? How would this work? And why doesn't the vulnerability description give answers to these questions?
While YAML has all sorts of issues and disadvantages compared to XML, security is certainly not one of them. XML is a crazy source of security issues by design, especially with the crazy idea of adding built-in support for URLs that parsers are expected to follow.
You use invisible characters whenever you press Enter or Space. If you're referring to Tab, many of the most popular programming languages like Go and Python use them as part of their syntax.
The reason YAML was popularized is because it was a response to XML which isn't user friendly to write. It's unfortunate that the spec got so convoluted, and uses a lot of implicit behavior, but I'd rather write YAML than XML, JSON or TOML for things like configuration files. Nowadays there might be better alternatives, but YAML is the de facto standard.
It's also unfortunate that YAML got abused by people who wanted to turn it into a DSL, so we ended up with thousands of lines of Ansible playbooks, CI workflows, and Helm charts, but here we are.
I have no horse in that race but just to see people talking about XML like this a quarter of a century after the first time I saw similar comments is just funny, I don't care who you are.
NIH syndrome and "inverse second system effect". In the real second system effect, the second system is more complicated because it includes everything that could possibly be perceived as missing in the first system. In the inverse second system effect the first system was perceived as too complicated, not too simple, so the second system is much simpler and doesn't do its job well.
It is tangentially related in that yaml became normal to use as a DSL within the devops world. As another post said, everything becomes a DSL eventually because people want to be "fully configurable" not realizing that is roughly the same thing as not being complete yet. But in this case the lack of direct acknowledgement of yaml as an interpreted language with an interpreter that doesn't think of itself as such and hence doesn't have a real sandbox, is what leads us to the present. People didn't use xml as a DSL as often because it was so flexible. That would be like using c++ as a DSL instead to write the interpreter for one.
Pretty cool and nice find. I already have a "malicious" Chart.yaml in mind for this attack just based on the description of vuln.
Fortunately, my dotfiles are managed with nix so trying to write to those files on a read only partition will raise many red flags for me.
I don't use bash, but maybe should write a dummy .bashrc (and other start up script equivalents for fish) as some sort of canary.
If I happen to overlook the malicious shell script crafted in a dependency on helm chart, I would get nasty errors that a process was trying to write to a read only file.
The original vulnerability description is not worded very well, here's my understanding of what's going on:
1. Attacker crafts a malicious Chart.yaml containing arbitrary code
2. Replaces Chart.lock with a symlink pointing to a sensitive file (like .bashrc or other startup scripts)
3. When you run helm dependency update, Helm processes the malicious Chart.yaml and writes the payload to whatever file the symlink targets
4. Code executes when the targeted file is next used (e.g., opening a new shell)
Why This Works: Helm follows the symlink during the dependency update process without validating the target, allowing arbitrary file writes outside the intended chart directory.
Can anyone explain in what setup an attacker who can create a symlink where Chart.lock was could not directly write .bashrc or similar? Is this related to how Git handles symlinks?
This has nothing to do with Git. A symlink can be packaged up in a tarball and shipped from one system to another. An attacker would need to create a malicious Chart.yaml file and a Chart.lock file pointing to another file. Then ship those to a system where dependencies are then updated.
This doesn't affect things like installing or upgrading a chart. Dependencies aren't updated at that time.
A symlink is just a special file that contains a string of text, it's not tightly bound to the target like a hard link. You can write anything into that string of text, including, say, "~/.bashrc". Then you can ship that symlink onto another system, and it suddenly points to your .bashrc.
Git just moves symlinks across systems as is, so yes, you can use git to deploy the exploit.
Helm is a program that allows users to creates packages which other users consume. Those packages contain files that are normally generated by Helm itself, but apparently if you alter your package definition by hand you can replace Chart.lock with a symlink.
As I'm typing this it's occurring to me that you probably shouldn't be able to do that. The fix they applied was to prevent the actual write from occurring when trying to write the lockfile and determining that the lockfile is a symlink. They could (should?) also validate that like, the package itself hasn't been screwed with in this manner.
Having read the CVE multiple times I am still unsure how 2. above happens? Is it possible through the malicious chart itself or is it a dependency for the CVE to be in play at all? And if the latter - what local process would write a symlink from a helm lock file to any kind of system start up script which doesn't point to a much bigger problem than this CVE?
The attacker creates a symlink (e.g., using `ln -s`) to another file. The attacker needs to create the malicious Chart.yaml file and symlink that the Chart.lock file points to.
If being able to create files and symlinks to them is a pre-condition for this, then it's not a serious security bug. If you have that kind of access then there are a million nefarious things you can do.
This is almost becoming a joke at this point, "assuming an attacker has access to the system, they can change things on the system".
thats funny because Helm refused to allow reference of external files (there is a github issue) but they follow symlinks xD
So the attack vector is:
- You have access to my file system
- You have access to the helm repository
You place malicious binaries outside the helm directory. Helm will now execute malicious code through the helm chart pointing outside the helm directory.
Don't I have already bigger problems if you have access to my file system to place there malicious code?
Is the danger here that one can get an execute permission? But if you can manipulate my helm chart why can you not also place the malicious code in the helm directory?
> You place malicious binaries outside the helm directory
No, helm is the one doing this part in the vuln. Chart.lock is made a symlink to some important file, and helm will happily write to it.
Yeah, there is a rather strong "downloading and executing arbitrary code from the Internet may lead to execution of arbitrary code" kind of vibe there.
Starting on the other side of the airtight hatchway: https://devblogs.microsoft.com/oldnewthing/20221004-00/?p=10...
Seems the normal mitigations apply i.e. validate with hash or save a local copy. Validate new versions before adopting
And yet you just described the behavior of many mid-size company "DevOps" departments.
That description seems really unclear, like how can `Chart.lock` be a symlink to a `.bashrc`?
Is the vulnerability that you ship a chart with `Chart.lock -> ../.bashrc`, and then helm writes to `Chart.lock`?
Why is the fix specific to Chart.lock (https://github.com/helm/helm/commit/76fdba4c8c2a4829a6b7abb4...), wouldn't the fix be instead that "A chart cannot contain any symlinks outside of its root"?
I think that there are "legitimate" use cases for symlinks that read from outside the root, which at this point are probably looked upon even less favorably. It's likely that making the change you're proposing would be backwards incompatible.
I agree that it's not clearly explained why this isn't a concern though. A cursory search for other instances of os.WriteFile doesn't seem to surface any thorough controls...
edit: ok actually it looks like the lockfile is special because it's the only instance of helm itself directly writing a file on behalf of a package consumer
What use-case?
If you have a chart that has `deploy.yaml` symlinked to `/home/john/testcharts/redis/deploy.yaml`, that chart is clearly not going to work on anyone's machine except john's, so that chart is useless on anyone else's machine.
If you're saying the use-case is for charts that aren't distributed, well, I'm saying we should ban all symlinks on distribution (downloading and unpacking a chart should fail if it has symlinks outside of the root), and I just can't imagine any use-case where a distributed chart with external symlinks makes sense.
If this whole thing is about charts that aren't distributed, but local to some developer's machine, well, in that case who cares if the developer can pwn themselves by typing "ln -s ~/.bashrc Chart.lock", they could have just pwned themselves by typing "bash" even more quickly.
I have use cases for linking Terraform lock files to keep various deployments/modules on consistent versions. I could see there being a use case for symlinking Chart.lock files although usually that's limited to an internal implementation and not something a general purpose chart would probably ship
i.e. you have 3 different charts that all depends on `cache`, `load balancer` and `database` charts and you want to only ever have 1 version deployed of those subcharts so you want the parent chart locks linked
Ya, I mean, I put "legitimate" in quotes for a reason. I think most people agree with you. This has been a thing that they've been aware of and struggling with for a while.
https://helm.sh/blog/2019-10-30-helm-symlink-security-notice...
Smattering an --allow-symlinks flag all over their commands seems to be the least inelegant way to handle this while still giving users an easy way to maintain compatibility. Maybe they'll come around to it after this.
Helm is an abomination, as the whole idea of using a text template engine to generate YAML is. And this vulnerability adds insult to injury ;)
Sorry, just can't really recover from trauma of counting spaces and messing up newlines, etc. when writing Helm templates. You know, Lisp "sucks" because "you need to count parenthesis" (you actually don't), yet Helm is a widely accepted technology where you need to count spaces for (n)indent ;)
What is the attack scenario here? Where are the security boundaries? How does the attacker gets their repository with a symlink in it to the victim? Is Helm typically run as a privileged user? How would this work? And why doesn't the vulnerability description give answers to these questions?
[dead]
As an aside, all these tools like aider, claude desktop ask for shell access to run codes.
Allowing LLMs to generate charts and what not via shell execution is a bad idea.
For a moment I thought it was about the synth…!
https://tytel.org/helm/
For a moment I thought it was about the Emacs package.
https://github.com/emacs-helm/helm
Likewise! Phew!
Although, the whole can of worms regarding synth/audio exploits is a pretty wild scene ..
If we're being honest, YAML is one of the dumbest ideas of the last 20 years to have proliferated. How we got from XML to here I cannot comprehend.
This is not the first RCE involving YAML and it won't be the last.
That was not RCE. It's not in yaml, it's in Helm's logic.
But glad you vented, I guess.
While YAML has all sorts of issues and disadvantages compared to XML, security is certainly not one of them. XML is a crazy source of security issues by design, especially with the crazy idea of adding built-in support for URLs that parsers are expected to follow.
Are we going to blame the next RCE we find in some application on XML just because that application uses XML somewhere?
If so, then I agree on blaming this on YAML.
Why we settled on a file format that relies on invisible characters I'll never know.
The gyrations people will go through to avoid using S-expressions...
You use invisible characters whenever you press Enter or Space. If you're referring to Tab, many of the most popular programming languages like Go and Python use them as part of their syntax.
The reason YAML was popularized is because it was a response to XML which isn't user friendly to write. It's unfortunate that the spec got so convoluted, and uses a lot of implicit behavior, but I'd rather write YAML than XML, JSON or TOML for things like configuration files. Nowadays there might be better alternatives, but YAML is the de facto standard.
It's also unfortunate that YAML got abused by people who wanted to turn it into a DSL, so we ended up with thousands of lines of Ansible playbooks, CI workflows, and Helm charts, but here we are.
It's unfortunate, but inevitable. Every structured text data format that sees widespread use, given enough time, will eventually be turned into a DSL.
In fact, once a structured text format is used as a data source for any process, it has already become a DSL.
i always enjoy writting json more. I feel it's easier to translate/integrate json into the code.
YAML is a superset of JSON, so go right ahead and write your .yml files in JSON.
Sometimes what makes something great is what it lacks. An automatic transmission, operator overloading, schema extensions, batteries etc.
[dead]
> many of the most popular programming languages like Go and Python use them as part of their syntax
Go doesn't use tabs or whitespace as a part of its syntax. It's a part of the formatting, but not the syntax of the language.
Python on the other hand, one extra tab or whitespace can cause havoc.
A search for XML on cve.org gives
> Showing 1 - 25 of 6,749 results for XML
Searching for YAML:
> Showing 1 - 25 of 288 results for YAML
Is that from the past two years?
I have no horse in that race but just to see people talking about XML like this a quarter of a century after the first time I saw similar comments is just funny, I don't care who you are.
In what way is this vulnerability YAML-specific?
NIH syndrome and "inverse second system effect". In the real second system effect, the second system is more complicated because it includes everything that could possibly be perceived as missing in the first system. In the inverse second system effect the first system was perceived as too complicated, not too simple, so the second system is much simpler and doesn't do its job well.
Also this vuln has nothing to do with YAML
It is tangentially related in that yaml became normal to use as a DSL within the devops world. As another post said, everything becomes a DSL eventually because people want to be "fully configurable" not realizing that is roughly the same thing as not being complete yet. But in this case the lack of direct acknowledgement of yaml as an interpreted language with an interpreter that doesn't think of itself as such and hence doesn't have a real sandbox, is what leads us to the present. People didn't use xml as a DSL as often because it was so flexible. That would be like using c++ as a DSL instead to write the interpreter for one.
Pretty cool and nice find. I already have a "malicious" Chart.yaml in mind for this attack just based on the description of vuln.
Fortunately, my dotfiles are managed with nix so trying to write to those files on a read only partition will raise many red flags for me.
I don't use bash, but maybe should write a dummy .bashrc (and other start up script equivalents for fish) as some sort of canary.
If I happen to overlook the malicious shell script crafted in a dependency on helm chart, I would get nasty errors that a process was trying to write to a read only file.
[dead]
But I thought security vulnerabilities couldn't happen in memory-safe languages!
But I thought accidents wouldn't happen if we wear helmets! Clearly they're worthless!
Sarcasm aside: wearing a helmet causes riders to take more risks, leading to more accidents.
https://www.sciencedirect.com/science/article/pii/S136984781...
I’d still wear one, but also try to be more careful knowing that the helmet provides a false sense of security.
I do believe the analogy holds very true with programming habits.
This isn't a memory bug.
Wondering how this kind of thing can be automatically discovered by an LLM. Anyone have any experience?
All the maintainers who get bombarded by LLM generated CVEs have a lot of experience with this.
Ask an LLM and find out