filesystems, processes & vendors.. oh my

Developer
Mar 20, 2009 at 5:32 PM
I started working on an implement of the iso9660 filesystem for CD-ROM's, and as I'm studying the FAT
code, I realized I would have to duplicate a lot of code. So, I started thinking about how to rework it
so that they could share code and my first inclination was that the VFS was a good place to put it.

However, as I thought about it some more, I realized that in order to do this "right", each FS implementation
needs to be it's own process so that the OS can run new FS's without having to recompile the kernel. This means
that VFS has to communicate with FS's through IPC. Therefore, I think I need to put my ISO9660 FS driver on
hold and help start work on process's and ipc. The problem is this is an area where I am weak. Can I get some
suggestions on how to begin work on that? Also, with regards to FS drivers specifically, I think that common
FS code needs to be implemented in a shared library to reduce excessive ipc ping-ponging between VFS and the
specific FS process. If this sounds like a good direction, I can start working on defining the interfaces
and ipc packets that would be needed to implement it "right".

This brings me to a long discussion that I had on IRC with tgiphil and illuminus. I see HUGE potential in
MOSA, perhaps a bit more than even founders of MOSA themselves. Allow me to illustrate. You can't start any
project without having a vision. Right now the vision is a driver framework so that a driver written for one
OS works on all others without having to be recompiled from source ( only JIT'd from the CIL ). Imagine for
a moment that MOSA becomes a useable desktop OS - not too far fetched I hope. As hardware manufacturers
start to support MOSA, they will realize that it is 10 to 100x easier to write drivers for MOSA than any
other system, and assuming we can deliver on performance, they will start to demand that Windows and Linux support
the MOSA driver framework. Furthermore, being a completely managed OS will free Intel and AMD up to
be more innovative with CPu design because they don't have to be backwards compatible any more. This will also
eventually allow other CPU vendors that don't currently compete for desktop processors to enter the market.
All they would have to do is develop a HAL and an AOT/JIT back-end for MOSA and they're in the game.

Why mention all this? I believe it's in every hardware manufacturer's best interest for MOSA to succeed. I
would like to propose that we contact some of the more progressive hardware vendors out there and pitch them
on helping us develop the framework. They would be able to help us make sure the framework is comprehensive.
Yes, it probably needs more of the basic framework before we can get far with them writing actual code, but I think
some of them may be willing to pitch in with helping define the functionality the framework needs to encompass
to meet their needs.

On another note, we talked about supporting embedded and industrial devices. These devices don't support things
like virtual memory, paging, and pre-emptiveness. However, they do have memory and i/o and a cpu instruction
set. To use PICs as an example, they don't even have a filesystem, therefore the "user" code has to be part of
the kernel. Because of this, we wouldn't even need to embed the JIT into the kernel for systems like that.
I'm not saying MOSA needs to support these kinds of build targets directly, but we may want to make sure the
design of the driver/device framework can support it.

Finally, I've been thinking that any modern OS needs to make driver installation seamless. There's no excuse for
drivers to be hard to find in the internet age. To that effect I think MOSA needs to come with all NIC drivers
possible, and to have an online clearinghouse that looks up PCI vendor ID's and connects you to the proper
driver. Manufacturers should host the driver, but there needs to be a central clearinghouse for locating them.
This wouldn't cost much and can eventually be supported by the hardware manufacturers if the driver framwork
gets popular enough.
Coordinator
Mar 22, 2009 at 1:16 AM
Hi Royce! 

> Therefore, I think I need to put my ISO9660 FS driver on hold and help start work on process's and ipc.

You are correct, we need to use IPC to communicate from application to the kernel and finally to the file system driver. IPC and how it will work is still in the early idea forming stage. Ideally, we'll like the kernel to setup the IPC channel between the application directly to the file system driver in some managed way. However, you do not have to hold off on the ISO9660 driver simply because IPC is unavailable. You can still design it and write a working implementation without. When IPC does become available only the interfaces (and more specifically how they work) and packaging will change. The driver will mostly likely stay the same.

> Why mention all this? I believe it's in every hardware manufacturer's best interest for MOSA to succeed.

I don't mean to sound negative here, but this has not been the case and unlikely will ever be. Even the popular Linux platform has a difficult time attracting active development support from hardware vendors. Regardless how easy it may be to write MOSA device drivers (and I'm not admitting it will be easy), hardware vendors have other goals. They want to make profit on the hardware and software they sell to large markets (consumers) or high profit margin niches (business). Nothing wrong with that - I support that completely. MOSA has other goals - to develop managed operating system. In addition, we may trade performance for safety and reliability. This may not appeal to some potential vendors or users. Anyways, MOSA is not a pancia for the problems with the device drivers.

> On another note, we talked about supporting embedded and industrial devices. These devices don't support things like virtual memory, paging, and pre-emptiveness.

We want to support embedded and industrial devices. We hope MOSA will allow for the drop in of almost any memory management design. At the moment, we are targeting x86 with virtual memory due to its popularity. However, feel free to design your own memory components which do not use virtual memory. You'll find support and assistance with this.
Coordinator
Mar 22, 2009 at 10:26 AM
Hi Royce,

Welcome! I'd like to split your post into some seperate topics as you're touching a lot of ground at once.

> I started working on an implement of the iso9660 filesystem for CD-ROM's, and as I'm studying the FAT
> code, I realized I would have to duplicate a lot of code. So, I started thinking about how to rework it
> so that they could share code and my first inclination was that the VFS was a good place to put it.

As the original designer of VFS I'd like to know where you see code duplication? I'm certain its not perfect,
but thinking about it (or: trying to remember what I did way back then) I don't think FS developers had to
duplicate a lot of code.

> However, as I thought about it some more, I realized that in order to do this "right", each FS implementation
> needs to be it's own process so that the OS can run new FS's without having to recompile the kernel. This means
> that VFS has to communicate with FS's through IPC.

Right and wrong. The kernel is an AppDomain like any other, it can load modules on demand. However this may
not be a good idea in order to secure the kernel and it's not the long-term goal. Ideally each (mounted) filesystem would
run its own process.

What people tend to forget about IPC is that .NET already provides rich IPC facilities with .NET Remoting - this
is the IPC we should use. .NET remoting imposes little additional work on the file system drivers, just that some objects
should derive from MarshalByRefObject. Opening the IPC channel will be handled by the kernel and the process stub
itself.

I'm not going to reply on the remainder of your  post here, as I think these are seperate topics you should bring up for
discussion.