[K42-discussion] RE: Possible benefit of running K42 on Cell CBE ...

[Elvis John Dowson]

Hi Jimi,
           What kind of functionality would a K42 SPU Linux Module contain,
vis-a-vis the standard K42 PPU Linux Module ?  

How big would it be and what is the bare minimum set of functionality that
is needed to run the K42 SPU Linux Module kernel on a CBE SPU ? 

What would the estimated size of a port of the K42 SPU Linux Kernel module
be, keeping in mind that there is only 256kb of local storage available per
SPE ? 

Is there some way to minimize the usage of SPU local storage by virtualizing
the SPU using the PPU and just incorporating minimal support for the SPU,
within the K42 PPU Linux Kernel, just to facilitate DMA transfers of program
and data using the SPE Memory Flow Controller (MFC) ?  Wouldnt that level of
support be sufficient support for the SPU by the K42 OS ?

Best regards,

Elvis Dowson

-----Original Message-----
From: Jimi Xenidis [mailto:jimix at watson.ibm.com] 
Sent: Wednesday, December 21, 2005 6:59 PM
To: Elvis John Dowson; Elvis John Dowson
Cc: Andrew Baumann; IBM Research K42 Discussion Forum; Orran Krieger; Andrew
Baumann; IBM Research K42 Discussion Forum
Subject: Re: Possible benefit of running K42 on Cell CBE ...

Ok, I like Orran do not want to turn away anyone working on k42  
especially with Cell.
Getting K42 to run on the PPE of a Cell processor should be a "cake  
walk".

However..

Using SPEs on k42 would require the _not_so_simple_ matter of forward- 
porting the K42-Linux module to a Linux that supports the BPA (BE)  
configuration --or-- back-porting the Linux BPA support to the Linux  
module K42 is currently running.  The later being a ridiculous exercise.
For the former, some K42 members are currently working to update the  
Linux module in K42, I'm not sure if that update includes the BPA  
changes or how hard it would be to continue the the new Linux module  
to one that supports the BPA config. Being that the SPU managment  
interface is /sysfs in Linux there may be more work then one would  
initially think.  I would caution that this would be more than a  
"side project".

more below.

On Dec 20, 2005, at 8:08 PM, Elvis John Dowson wrote:
>         The initial idea for taking a look at K42 for the Cell, was  
> to see if can be used to support dynamic management via addition  
> and removal of Cell Computational Nodes from a Cell Cluster, and to  
> see if data can be shared and access via the individual SPUs, using  
> a shared memory model.
hmm, so you are looking to use K42 to share memory across Cell nodes?
What makes you think that K42 is currently useful for sharing memory  
between cluster nodes?

> I posted a query regarding support for this type of operation, in  
> the hardware, on the cell forum and they said that the Element  
> Interconnect Bus on the Cell processor can be configured for  
> operation in a coherent mode, but would require the additional  
> support of a coherent memory switch. http://www-128.ibm.com/ 
> developerworks/forums/dw_thread.jsp?forum=739&thread=102913&cat=46
The EIB connects all the processing, IO(BIC) and memory(MIC) elements  
in the the chip.

When 2 (this implementation only supports 2) CBEs are connected by  
the BE Interface Unit (BEI), the BEI can be configured to form a  
fully snooping 2 way SMP bus, or a 2 node "cluster" with hi-speed IO  
bus interconnect that is also cache coherent.

Re: the "coherent memory switch". I'm not sure where this comes from,  
but I think Dan is just talking about standard cluster interconnects  
and not HW specifically for the CBE. I'd be pleasantly surprised to  
learn that such a switch is being built be someone. But, we can all  
wait for his reply :)
> Maybe, from a K42 perspective, there may not be much to do SPU  
> specific and the host PPU could co-ordinate all I/O communications,  
> but if an SPU were to reference a DMA memory operation (queue/ 
> request a read or write) to the local storage of another SPU, in a  
> computational grid, I guess some level of O/S support should exist.
It is the responsibility of the SW on the PPE to provide the  
translations for the DMA units (MFC).
This support is in linux today, along with a library to drive it.

If that target memory is on another CBE then the PPE would have to  
make the appropriate IO Translations to make sure that the memory  
area is mapped.  If the later mapping is static, the FW can create a  
"standard" mapping, which it does.

How a "coherent memory switch" is configured for another node would  
be additional SW.

.Now to dissect the following...
> The initial work on UML, I plan to do is to target specific SPU  
> code generation from specific UML Model Elements that have been  
> specifically tagged and stereotype for the SPU.
Here you plan to create an SPU/UML model so that you can generate C/C+ 
+ code for the SPU compiler, correct?
Will your results be self-contained or will they require more from  
the PPU size other than loading and setting up DMA addressibility?
If they are self contained there are ways (or there used to be) to  
use systemsim to just run the SPU part.

> I have already tested the UML automatic code generation framework  
> for the PPU and it works.
By "works" you mean it generates code and the code compiles and links?
Is there anything Cell specific about the generated code?
Can't you just use a standard LinuxPPC system? Since your code is 32  
bit, your HW requirements are minimal.
> I'm just waiting for simulator network support for the Cell  
> Simulator so that the PPU application can be debugged at the model  
> level,
..Assuming you _do_ have a Cell dependency...
Oh, Dude! this smells _way_ wrong.  From what I understand, all you  
want is for an app to have a character driven communication channel  
to the machine running the simulator.  Correct?
Yes, with HW network is best, but there are _far_ more efficient ways  
to do this with systemsim then simulating an entire network stack.

Anyway, I would strive to make the communications channel as  
transport independent as possible so you can switch the underlying  
transport for different environments (sim-channel, TCP/IP, Shared  
memory, etc)

> for example,
>
> real-time animation of the UML sequence diagram,
> to show external message communication between active objects in  
> the system,
> the state of the threads running in the system,
> dynamic UML state-charts that show the current state of the  
> individual objects in the system,
> plus inspection of object states at the UML model level from within  
> the model browser and inspect the current state of the object  
> attributes at the UML model level, at run-time

Cool.
> Note that GDB is not required for debugging and animating the UML  
> model; you are effectively debugging and animating the model at the  
> design level. If you find an execution trace that is not right or  
> an object struck at a particular state, you it would be possible to  
> try to launch a GDB session and debug the code at the code /  
> assembly level, concurrently with the UML model, being debugged at  
> the design level. I have been working in this manner for all the  
> projects that I have worked on, in my previous companies (British  
> Aerospace and Snecma Aerospace) for both real-time asynchronous  
> (UML) and real-time synchronous (SCADE) model driven development  
> environment for real-time embedded avionics safety critical systems  
> and desktop scientific engineering application development.
>
>
>
> Right now I'm attempting to define a CBE UML Profile, for the Cell  
> Processor, the outline steps can be found here in this post :  
> http://www-128.ibm.com/developerworks/forums/dw_thread.jsp? 
> forum=739&thread=102743&cat=46
hmm, since we have no immediate plans to create a K42/Cell  
programming model, the first hack on K42 would simply allow the Linux/ 
BPA SW stack to run on K42, in the same way that the Linux user app  
stack runs today.  So, at least to me, extending your favorite POSIX/C 
++/UML models to perform the PU-to-SPU transitions using the Cell SDK  
makes a whole lot more sense.
>
>
> At some stage, after I finish this work, I would like to  
> investigate the K42 O/S to see if can be used to support the  
> development of a dynamically reconfigurable and scalable  
> computational cluster, identify the specific portions that would  
> need to be adapted and estimate the effort for this adaptation.
Now this would be cool.
> Having a UML model representation of the K42 sources would have  
> helped, first in being able to understand the O/S, and second, in  
> being able to completely generate the code from the UML models and  
> debug it at the model level.
Understand, IMNSHO, we are kernel developers using C++ to solve a lot  
of tedious programming issues.  We generally are not in favor of  
"generated" code and tho' there is use of C++ Templates their  
usefulness is arguable.

> The same techniques that I have outlined in the dW post about the  
> CBE Profile can be adapted to work for creating specific C/C++  
> emitted code templates using an automatic code-generator that will  
> parse a K42 specific UML Meta-Model. This meta-model, when used  
> along with the C/C++ ( or virtually any programming language,  
> including assembly ) code generation rules, can be applied to a UML  
> model created and code automatically generated for that UML model.
>
>
Thanks for all this info.

>
>
> Best regards,
>
>
>
> Elvis Dowson
>
>
>
>
>
> From: Orran Y Krieger [mailto:okrieg at us.ibm.com]
> Sent: Wednesday, December 21, 2005 12:12 AM
> To: Elvis John Dowson
> Cc: 'Andrew Baumann'; 'Jimi Xenidis'; k42-discussion at ozlabs.org
> Subject: RE: [K42-discussion] What is the correct version of gcc,  
> glibc, binutils, linux and linuxthreads required for a successful  
> K42 build ?
>
>
>
>
>
>
>
> >> I don't really understand why the primary thing you are focused  
> on, i.e., having a way to generate SPU code from UML really relates  
> to K42.  I don't object, mind you, for people to use K42, I love  
> it, but I hate to add pure overhead to your primary goal?
>
>
>
>

-JX
--
"I got an idea, an idea so smart my head would explode if I even
   began to know what I was talking about." -- Peter Griffin (Family  
Guy)