28 comments

[ 427 ms ] story [ 1306 ms ] thread
One totally practical use case for this is to ensure that none of the virtual machines that you have allocated on your cloud provider are on the same physical hardware. This would be extremely useful for reliability purposes. I wonder how hard it would be to build something like that on top of this library?
not that hard, but keep in mind the current limitations on supported cpus
Interesting idea. Don't think that would work for multi-socket boards though, which I think are most servers.
Ah, didn't see the restriction to single socket boards but that makes sense. Probably not going to work for any of the major cloud providers then.
It could work intermittently depending on which vCPUs your VM or covert channel apps get assigned (intermittently as that could change unless your hypervisor uses CPU pinning...)
Given that it's not economical for cloud providers to let too many of their customers monkey with resource provisioning, I gather such efforts on the part of the customer can only be so effective.
With virtual machines I control, I don't have to use covert channels?

https://cseweb.ucsd.edu/~hovav/dist/cloudsec.pdf (2009)

Edit: we describe an empirical mea-surement study focused on understanding VM placement in the EC2 system and achieving co-resident placement for an adversary. To do this, we make use of network probing both to identify public services hosted on EC2 and to provide evidence of co-residence (that two instances share the same physical server). In particular, we utilize nmap, hping, and wget to perform network probes to determine liveness of EC2 instances

The hard disk contention channel was used in Section 6 for establishing co-residence of instances.

That assumes the provider will disclose hardware info to you, which is rarely the case.
(I apologize in advance if I've missed recognition of your authority on this topic; as an outsider it appears you are wrong.)

Are you speaking generally or specifically addressing a shortcoming of the linked paper which demonstrates a methodology to determine whether or not two AWS VMs are on the same physical host?

After a brief review I still don't see any mention of "hardware info" you mention as assumed being available.

Edit: If something has come up since 2009 with this specific paper, I'd appreciate the heads-up!

I don't know that it would be difficult to reproduce the test that is mentioned in the paper linked below, but they do mention the following anyways for EC2:

> A single account was never seen to have two instances simultaneously running on the same physical machine, so running n instances in parallel under a single account results in placement on n separate machines

I noticed the paper tested an Xeon E5-2670 on EC2, which is old (Sandy Bridge; eg, m2's) and pre-dates Intel Cache Allocation Technology (IntelCAT):

https://software.intel.com/en-us/articles/introduction-to-ca...

Support for IntelCAT was added to Xen two years ago. Eg:

    [root@vmm-psr vmm]# xl psr-hwinfo --cat
    Cache Allocation Technology (CAT):
    Socket ID       : 0
    L3 Cache        : 12288KB
    Maximum COS     : 15
    CBM length      : 12
    Default CBM     : 0xfff
And to Linux 4.10 a couple of months ago: https://lwn.net/Articles/694800/

While I suspect IntelCAT could disable this side channel, I'd personally hesitate before turning it on for everyone on the cloud. Does it out weigh the performance cost? ... Can customers opt-in to this?

If there are no more shared cache sets between VMs, then yes, it's dead ofc

I would suspect if one VM would like to be separated from the others in terms of cache sets, the others really don't have the choice to opt out?

Yes it was specifically called out in someone's tweet I'm trying to find but I believe this was mentioned in the paper.

Edit: My mix-up on the paper!

Malware Guard Extension: Using SGX to Conceal Cache Attacks (Extended Version) | https://arxiv.org/pdf/1702.08719.pdf (2017)

Recently, Intel introduced an instruction set extension called CAT (cache allocation technology) [51]. With Intel CAT it is possible to restrict CPU cores to one of the slices of the last-level cache and even to pin cache lines. Liu et al. [83] proposed a system that uses CAT to protect general purpose software and cryptographic algorithms. Their approach can be directly applied to protect against a malicious enclave. However, this approach also does not allow to protect enclaves from an outside attacker.

[...]

Instead of using Intel CAT on the operating level it could also be used to protect enclaves on the hardware level. By changing the eenter instruction in a way that it implicitly activates CAT for this core, any cache sharing between SGX enclaves and the outside as well as co-located enclaves could be eliminated. Thus, SGX enclaves would be protected from outside attackers. Furthermore, it would protect co-located enclaves as well as the operating system and user programs against malicious enclaves.

[83] https://www.researchgate.net/profile/Yuval_Yarom/publication...

Does this have any practical applications? You might get a perf win, but the likelihood of being collocated is small.
I think at this point it's mostly of interest to security researchers and malware writers. It's not hard to imagine malware using this technique to transfer data out of an infected VM in a difficult to detect manner. If you can infect enough VMs it might be worth it. Also, this type of technique could be interesting to government employed malware writers.
Another interesting way to exfiltrate data is via DNS. Few networks block DNS, so you can set up a protocol on your own DNS server to transfer data.

The bandwidth is pretty decent, too.

I'm still surprised how well this classic attack works. I've seen $100k IDS and firewall packages not detect this.

I use this personally. Works behind captive portals sometimes too: http://code.kryo.se/iodine/

Reminds me a lot of this DEF CON talk: https://www.youtube.com/watch?v=7X772EBdvnM

He managed to stream video across VMs by doing a similar cache timing trick, which brought a round of applause from the crowd. Very cool stuff.

I guess you could call it an evolution, because memory deduplication is not always enabled anymore (research like this might have played a role here ;)

he talks about that @35:27 in the video