[coyotos-dev] EROS small processes

[Eric Northup]

Jonathan S. Shapiro wrote:
> Small spaces are a nuisance to implement. A while back AMD implemented a
> mechanism that significantly improved TLB flush performance, and in
> later processors Intel was forced to follow AMD's lead to keep
> competitive. It isn't clear how important the small space optimization
> is for these newer processors.

Due to errata and its general lack of improvement for common, 
Windows-based workloads, the TLB flush filter on AMD64 processors is 
often disabled by the BIOS.  This is unfortunately, because the final 
page of 
http://i30www.ira.uka.de/teaching/coursedocuments/130/mkc-07-smalladdressspaces.pdf 
notes that AMD's flush filter reduced L4 Pistachio IPC time by a factor 
of 3 (750-> 240 cycles).


AMD CPU's EFER register, bit 13 ("Long Mode Segment Limit Enable") seems 
like it would allow implementing small spaces even for 64-bit code.  It 
is under-documented in the publicly available manuals, but seems to have 
added at VMware's request.

The recent AMD parts do penalize code running with segment bases -- both 
data and instruction fetches through segments with nonzero bases become 
slower.  Data loads/stores take an extra cycle and code fetches take an 
extra 2.

For processes more concerned with the request<->reply latency than 
computational throughput, that penalty likely doesn't matter.


As of March 2006, Andy Glew (chip architect at Intel/AMD/Intel) claimed

> No slowdown on present Intel CPUs.
>
> 1 cycle slowdown on AMD CPUs (K7 and K8) for nonzero base.

Also, the AMD software optimization guide states that mispredicted
branches incur additional latency (12 vs 10 cycles) if the code segment
has a nonzero base.

-Eric