Discussion:
Open source x86 emulator anywhere?
(too old to reply)
Vir Campestris
2022-03-21 21:20:54 UTC
Permalink
This is something I've thought about for a while.

I have an old computer here which runs a '286 in real address mode with
an off-chip memory manager. I have fond memories of the project, and
would like to get an emulator of it running.

But I'm not about to write a '286 emulator from the ground up. I tried
it once for an 8080, and that was painful enough - it never worked
properly even though I could just about get CP/M to run DDT.

Is there a decent open source '286 emulator, ideally written in C++
although C would do fine, that I could use to build my emulator around?

Andy
J. Clarke
2022-03-21 21:57:13 UTC
Permalink
On Mon, 21 Mar 2022 21:20:54 +0000, Vir Campestris
Post by Vir Campestris
This is something I've thought about for a while.
I have an old computer here which runs a '286 in real address mode with
an off-chip memory manager. I have fond memories of the project, and
would like to get an emulator of it running.
But I'm not about to write a '286 emulator from the ground up. I tried
it once for an 8080, and that was painful enough - it never worked
properly even though I could just about get CP/M to run DDT.
Is there a decent open source '286 emulator, ideally written in C++
although C would do fine, that I could use to build my emulator around?
There are three x86 emulators with source code available that I am
aware of. Bochs starts at 80386--what would have to be done to it to
make it accurately emulate an 80286 I have no idea. PCEmu and
IBMUlator both claim to emulate 80286 but with what accuracy I have no
idea. They both emulate specific systems and require ROM images, so
may not be what you want. OTOH you might be able to beat one of them
into what you need.
meff
2022-03-21 22:03:28 UTC
Permalink
Post by Vir Campestris
Is there a decent open source '286 emulator, ideally written in C++
although C would do fine, that I could use to build my emulator around?
Andy
Does [QEMU](https://www.qemu.org/) work for your needs?
Ahem A Rivet's Shot
2022-03-21 22:19:21 UTC
Permalink
On Mon, 21 Mar 2022 21:20:54 +0000
Post by Vir Campestris
This is something I've thought about for a while.
I have an old computer here which runs a '286 in real address mode with
an off-chip memory manager. I have fond memories of the project, and
would like to get an emulator of it running.
In addition to the ones jclarke mentioned there's dosbox
(dosbox.sourceforge.net) that emulates a 286 or 386 PC in real or protected
mode.
--
Steve O'Hara-Smith
Odds and Ends at http://www.sohara.org/
Andreas Kohlbach
2022-03-21 22:36:53 UTC
Permalink
Post by Vir Campestris
This is something I've thought about for a while.
I have an old computer here which runs a '286 in real address mode
with an off-chip memory manager. I have fond memories of the project,
and would like to get an emulator of it running.
But I'm not about to write a '286 emulator from the ground up. I tried
it once for an 8080, and that was painful enough - it never worked
properly even though I could just about get CP/M to run DDT.
Is there a decent open source '286 emulator, ideally written in C++
although C would do fine, that I could use to build my emulator around?
MAME emulates any computer using a microprocessor as CPU. No idea if it
allows you to only emulated a 286. But you can emulate the IBM PC/AT which
sports a 80286. You only need the BIOS rom (which I happen to have :-)
and floppy or hard disk images with the software you intend to run.

<https://www.mame.net/>

No idea what a "off-chip memory manager" is though.

emulating IBM 5150 on MAME
running CP/M-86.
--
Andreas

https://news-commentaries.blogspot.com/
Vir Campestris
2022-03-26 17:19:30 UTC
Permalink
Post by Andreas Kohlbach
MAME emulates any computer using a microprocessor as CPU. No idea if it
allows you to only emulated a 286. But you can emulate the IBM PC/AT which
sports a 80286. You only need the BIOS rom (which I happen to have:-)
and floppy or hard disk images with the software you intend to run.
<https://www.mame.net/>
No idea what a "off-chip memory manager" is though.
http://youtu.be/eF-uFIN06Ck emulating IBM 5150 on MAME
running CP/M-86.
Thanks all for the pointers. I'll have a look when I get time. Which
ought to be in May, when I've retired!

Off-chip memory manager - well, you all know that when the '286 came out
it was an attempt to get a bigger address space on top of the 8086 which
was limited to 1MB.

Except Intel did it in such a way almost nobody could use it. OS/2 was
about the only one I know of, and that wouldn't really run 8086 programmes.

What we did was to take the top address bits and use them to index into
a fast RAM; the output of the RAM became new address bits.

More address bits.

Which means that within the scope of one process you have the full 1MB
address space of the 8086, but the computer can run more than one
process, each with their own address space. You can even do proper
paging if you want. Sadly virtual addressing wasn't possible; there was
no way to restart the current instruction after a page fault.

Concurrent CP/M was designed with such systems in mind.

Andy
J. Clarke
2022-03-27 01:18:54 UTC
Permalink
On Sat, 26 Mar 2022 17:19:30 +0000, Vir Campestris
Post by Vir Campestris
Post by Andreas Kohlbach
MAME emulates any computer using a microprocessor as CPU. No idea if it
allows you to only emulated a 286. But you can emulate the IBM PC/AT which
sports a 80286. You only need the BIOS rom (which I happen to have:-)
and floppy or hard disk images with the software you intend to run.
<https://www.mame.net/>
No idea what a "off-chip memory manager" is though.
http://youtu.be/eF-uFIN06Ck emulating IBM 5150 on MAME
running CP/M-86.
Thanks all for the pointers. I'll have a look when I get time. Which
ought to be in May, when I've retired!
Off-chip memory manager - well, you all know that when the '286 came out
it was an attempt to get a bigger address space on top of the 8086 which
was limited to 1MB.
Except Intel did it in such a way almost nobody could use it. OS/2 was
about the only one I know of, and that wouldn't really run 8086 programmes.
Novell and AT&T did it. SVR2 (I think it was 2) would run on a '286
and address 16 meg if installed. Novell Advanced Netware/286 could
even run a DOS session.
Post by Vir Campestris
What we did was to take the top address bits and use them to index into
a fast RAM; the output of the RAM became new address bits.
More address bits.
Which means that within the scope of one process you have the full 1MB
address space of the 8086, but the computer can run more than one
process, each with their own address space. You can even do proper
paging if you want. Sadly virtual addressing wasn't possible; there was
no way to restart the current instruction after a page fault.
Concurrent CP/M was designed with such systems in mind.
Andy
Vir Campestris
2022-03-27 20:40:09 UTC
Permalink
Post by J. Clarke
Novell and AT&T did it. SVR2 (I think it was 2) would run on a '286
and address 16 meg if installed. Novell Advanced Netware/286 could
even run a DOS session.
I'd forgotten about Netware (even though I once went to Provo to work on
a bugfix with them) and I'd clean forgotten about the DOS mode.

IIRC it put the chip back into real address mode while it ran, using the
keyboard controller reset line to reset the processor. But it's a long
time ago.

I didn't know about SVR2.

Andy
Bob Eager
2022-03-27 21:21:01 UTC
Permalink
Post by Vir Campestris
Post by J. Clarke
Novell and AT&T did it. SVR2 (I think it was 2) would run on a '286
and address 16 meg if installed. Novell Advanced Netware/286 could
even run a DOS session.
I'd forgotten about Netware (even though I once went to Provo to work on
a bugfix with them) and I'd clean forgotten about the DOS mode.
IIRC it put the chip back into real address mode while it ran, using the
keyboard controller reset line to reset the processor. But it's a long
time ago.
I didn't know about SVR2.
Did anyone mention PC/IX? I have a book on that, that a colleague wrote.
--
Using UNIX since v6 (1975)...

Use the BIG mirror service in the UK:
http://www.mirrorservice.org
John Levine
2022-03-27 22:32:50 UTC
Permalink
Post by Bob Eager
Post by Vir Campestris
Post by J. Clarke
Novell and AT&T did it. SVR2 (I think it was 2) would run on a '286
and address 16 meg if installed. Novell Advanced Netware/286 could
even run a DOS session.
I'd forgotten about Netware (even though I once went to Provo to work on
a bugfix with them) and I'd clean forgotten about the DOS mode.
IIRC it put the chip back into real address mode while it ran, using the
keyboard controller reset line to reset the processor. But it's a long
time ago.
I didn't know about SVR2.
Did anyone mention PC/IX? I have a book on that, that a colleague wrote.
PC/IX ran in real mode on an 8086 or 286. I wrote small bits of it. Interactive Systems,
which wrote PC/IX, never wrote a protected mode 286 system.

There were versions of Xenix that ran in 286 protected mode.
--
Regards,
John Levine, ***@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly
Ahem A Rivet's Shot
2022-03-28 19:17:41 UTC
Permalink
On Sun, 27 Mar 2022 21:40:09 +0100
Post by Vir Campestris
IIRC it put the chip back into real address mode while it ran, using the
keyboard controller reset line to reset the processor. But it's a long
time ago.
That was the hardware mechanism provided for exiting protected mode
on the AT - everything based 286 used it.
--
Steve O'Hara-Smith
Odds and Ends at http://www.sohara.org/
Johnny Billquist
2022-03-28 08:47:53 UTC
Permalink
Post by Vir Campestris
Thanks all for the pointers. I'll have a look when I get time. Which
ought to be in May, when I've retired!
Off-chip memory manager - well, you all know that when the '286 came out
it was an attempt to get a bigger address space on top of the 8086 which
was limited to 1MB.
Except Intel did it in such a way almost nobody could use it. OS/2 was
about the only one I know of, and that wouldn't really run 8086 programmes.
What we did was to take the top address bits and use them to index into
a fast RAM; the output of the RAM became new address bits.
More address bits.
Which means that within the scope of one process you have the full 1MB
address space of the 8086, but the computer can run more than one
process, each with their own address space. You can even do proper
paging if you want. Sadly virtual addressing wasn't possible; there was
no way to restart the current instruction after a page fault.
If you can't restart an instruction, you can't do paging. You'll have to
swap all memory in before execution, so no page faults happen.
Virtual memory is fine though (I think you reversed what paging and
virtual memory is :-) ).

But yeah, I think pretty much noone every used the 286 much, for these
reasons.

Johnny
Vir Campestris
2022-03-28 20:40:31 UTC
Permalink
Post by Johnny Billquist
Post by Vir Campestris
Thanks all for the pointers. I'll have a look when I get time. Which
ought to be in May, when I've retired!
Off-chip memory manager - well, you all know that when the '286 came
out it was an attempt to get a bigger address space on top of the 8086
which was limited to 1MB.
Except Intel did it in such a way almost nobody could use it. OS/2 was
about the only one I know of, and that wouldn't really run 8086 programmes.
What we did was to take the top address bits and use them to index
into a fast RAM; the output of the RAM became new address bits.
More address bits.
Which means that within the scope of one process you have the full 1MB
address space of the 8086, but the computer can run more than one
process, each with their own address space. You can even do proper
paging if you want. Sadly virtual addressing wasn't possible; there
was no way to restart the current instruction after a page fault.
If you can't restart an instruction, you can't do paging. You'll have to
swap all memory in before execution, so no page faults happen.
Virtual memory is fine though (I think you reversed what paging and
virtual memory is :-) ).
I think we're disagreeing on terminology.

You can have lots of pages on a '286 with the MMU we built. What you
can't have is a page fault. Which means you can't have virtual memory -
everything the app needs must be in real RAM all the time.
Post by Johnny Billquist
But yeah, I think pretty much noone every used the 286 much, for these
reasons.
And we used it just as a faster 8086.

Andy
Johnny Billquist
2022-03-29 12:58:43 UTC
Permalink
Post by Vir Campestris
Post by Johnny Billquist
Post by Vir Campestris
Thanks all for the pointers. I'll have a look when I get time. Which
ought to be in May, when I've retired!
Off-chip memory manager - well, you all know that when the '286 came
out it was an attempt to get a bigger address space on top of the
8086 which was limited to 1MB.
Except Intel did it in such a way almost nobody could use it. OS/2
was about the only one I know of, and that wouldn't really run 8086
programmes.
What we did was to take the top address bits and use them to index
into a fast RAM; the output of the RAM became new address bits.
More address bits.
Which means that within the scope of one process you have the full
1MB address space of the 8086, but the computer can run more than one
process, each with their own address space. You can even do proper
paging if you want. Sadly virtual addressing wasn't possible; there
was no way to restart the current instruction after a page fault.
If you can't restart an instruction, you can't do paging. You'll have
to swap all memory in before execution, so no page faults happen.
Virtual memory is fine though (I think you reversed what paging and
virtual memory is :-) ).
I think we're disagreeing on terminology.
Very likely. But I'll do one attempt at "converting" you. :-)
Post by Vir Campestris
You can have lots of pages on a '286 with the MMU we built. What you
can't have is a page fault. Which means you can't have virtual memory -
everything the app needs must be in real RAM all the time.
Pages basically means the virtual memory is not necessarily all
contiguous in physical memory. I think we can agree on that part. It
also means that you can have a setup where not all of your virtual
memory are mapped somewhere into physical memory at some point in time.
Paging is the action of reading in a page after a page fault. A page
fault happens when a page is not resident in memory.

I do think we can agree on things so far. Possibly you might have some
other definition of what paging is, which would be interesting to hear,
since you think you can do paging.

But the usual definition of paging is that you read/write individual
pages as needed. Which is different than swapping, which is dealing with
all the process memory in one go. With swapping you are either in
memory, or not. You swap a whole process in or out. You don't swap
individual pages. Then we're back to paging again. But even when you
have pages, swapping as such can take place. Nothing says that swapping
cannot be done on an address space that is divided into pages.

Now, virtual memory is a completely different thing. Virtual memory is,
as the name suggest, memory which is virtual, ie. not real/physical. And
it's not real in the sense that the address space your program have
might have an address 0, and you might have some data there. Another
process might also have an address 0, and some data there. But this is
not the same address 0, and it's not the same data. Both processes
thinks they have a bunch of memory which is their own, and it acts like
memory, and can be used. But in the actual machine, there is just
physical memory, and there is an address 0, but it's not the address, or
the data either of these processes are thinking about, referring to, or
using. Each process have its own virtual memory. From time to time, that
virtual memory might map to physical memory, but that might not always
be the case. Further, the actual physical memory address used for
virtual 0 might be anything. And for each process, virtual 0 might map
to a different physical address.
But the point is, from the process point of view, it appears that he has
a full address space with memory, and it's his alone, he can do anything
in there, and it's completely isolated, and the address space as such is
private. He might not even see the OS memory, even though it obviously
also do exist.

The size of physical memory compared to virtual memory is actually
irrelevant. However, if you want to have more virtual memory than there
is physical memory, not only do you need virtual memory, but you also
need paging. Because without paging, you cannot exceed physical memory.
But you can still have virtual memory.

In you head, I bet you are thinking that virtual memory means paging.
But as you observed yourself, you cannot even have page faults if you
can't restart instructions, so at that point your process cannot have
more virtual memory than physical memory. Does that then mean you
actually do not have virtual memory either. You obviously do not have
page faults. What does "paging" mean for you then?

And to make some practical examples: the PDP-11 have a 16 bit virtual
address space, but a 22-bit physical address space. It divides the
16-bit address space into 8 pages, and you can even restart
instructions. So in theory you could do a system with paging, but as far
as I know, none have been done. However, you do have virtual addressing,
and virtual memory in pretty much all operating systems that are
multiuser ones. (They all just do swapping. But total memory used by all
processes might easily exceed the available physical memory, but a
single process have to have all it's memory in physical space before
executing. I could go into more details why this is, if anyone is
interested.)
But in your view then, do the PDP-11 have or not have virtual memory?
Would it change if you were to implement demand paging? Or will it never
have virtual memory, no matter what, just because of the basic fact that
it has a larger physical address space than virtual? After all, apart
from that detail, it would be doing exactly the same thing as another
computer you would claim do have virtual memory.

Similar on VAX. The original VAX had a 32-bit virtual address space, but
a 30-bit physical one. So you would think/argue that this machine then
have virtual memory, as well as paging (well, not all operating systems
on the VAX did paging. Classical example is the first Unix port, which
only did swapping on the VAX, and did not do any demand paging - did it
not have virtual memory then?). However, the last revision to the VAX
architecture allowed the physical address space to go up to 34 bits. So
in theory you could then see a machine with more physical memory than
virtual memory could address. Does this then mean that the VAX no longer
have virtual memory? Because of that extension? Or would it just stop
having virtual memory the moment you actually inserted that much
physical memory?

Sorry for such a long post. It's just that it always bothers me when
people use virtual memory for a term that is paging, and think that
virtual memory can only exist if the virtual address space is larger
than the physical address space. It leads to funny and strange
conclusions, and redefinition of machines in some situations, as well as
disagreeing with age old documented usage of the terms. Check any PDP-11
processor handbook for virtual addresses. It's in there.
Post by Vir Campestris
Post by Johnny Billquist
But yeah, I think pretty much noone every used the 286 much, for these
reasons.
And we used it just as a faster 8086.
Yeah. That might have been a popular use. :-D

Johnny
Vir Campestris
2022-03-30 20:49:47 UTC
Permalink
Post by Johnny Billquist
Post by Vir Campestris
Post by Johnny Billquist
Post by Vir Campestris
Thanks all for the pointers. I'll have a look when I get time. Which
ought to be in May, when I've retired!
Off-chip memory manager - well, you all know that when the '286 came
out it was an attempt to get a bigger address space on top of the
8086 which was limited to 1MB.
Except Intel did it in such a way almost nobody could use it. OS/2
was about the only one I know of, and that wouldn't really run 8086
programmes.
What we did was to take the top address bits and use them to index
into a fast RAM; the output of the RAM became new address bits.
More address bits.
Which means that within the scope of one process you have the full
1MB address space of the 8086, but the computer can run more than
one process, each with their own address space. You can even do
proper paging if you want. Sadly virtual addressing wasn't possible;
there was no way to restart the current instruction after a page fault.
If you can't restart an instruction, you can't do paging. You'll have
to swap all memory in before execution, so no page faults happen.
Virtual memory is fine though (I think you reversed what paging and
virtual memory is :-) ).
I think we're disagreeing on terminology.
Very likely. But I'll do one attempt at "converting" you. :-)
Post by Vir Campestris
You can have lots of pages on a '286 with the MMU we built. What you
can't have is a page fault. Which means you can't have virtual memory
- everything the app needs must be in real RAM all the time.
Pages basically means the virtual memory is not necessarily all
contiguous in physical memory. I think we can agree on that part. It
also means that you can have a setup where not all of your virtual
memory are mapped somewhere into physical memory at some point in time.
Paging is the action of reading in a page after a page fault. A page
fault happens when a page is not resident in memory.
I do think we can agree on things so far. Possibly you might have some
other definition of what paging is, which would be interesting to hear,
since you think you can do paging.
But the usual definition of paging is that you read/write individual
pages as needed. Which is different than swapping, which is dealing with
all the process memory in one go. With swapping you are either in
memory, or not. You swap a whole process in or out. You don't swap
individual pages. Then we're back to paging again. But even when you
have pages, swapping as such can take place. Nothing says that swapping
cannot be done on an address space that is divided into pages.
Now, virtual memory is a completely different thing. Virtual memory is,
as the name suggest, memory which is virtual, ie. not real/physical. And
it's not real in the sense that the address space your program have
might have an address 0, and you might have some data there. Another
process might also have an address 0, and some data there. But this is
not the same address 0, and it's not the same data. Both processes
thinks they have a bunch of memory which is their own, and it acts like
memory, and can be used. But in the actual machine, there is just
physical memory, and there is an address 0, but it's not the address, or
the data either of these processes are thinking about, referring to, or
using. Each process have its own virtual memory. From time to time, that
virtual memory might map to physical memory, but that might not always
be the case. Further, the actual physical memory address used for
virtual 0 might be anything. And for each process, virtual 0 might map
to a different physical address.
But the point is, from the process point of view, it appears that he has
a full address space with memory, and it's his alone, he can do anything
in there, and it's completely isolated, and the address space as such is
private. He might not even see the OS memory, even though it obviously
also do exist.
The size of physical memory compared to virtual memory is actually
irrelevant. However, if you want to have more virtual memory than there
is physical memory, not only do you need virtual memory, but you also
need paging. Because without paging, you cannot exceed physical memory.
But you can still have virtual memory.
In you head, I bet you are thinking that virtual memory means paging.
But as you observed yourself, you cannot even have page faults if you
can't restart instructions, so at that point your process cannot have
more virtual memory than physical memory. Does that then mean you
actually do not have virtual memory either. You obviously do not have
page faults. What does "paging" mean for you then?
I can have a bunch of discontinuous pages in memory which appear to be
contiguous to the process running.

I can add individual pages to a process as it requests. A small process
doesn't have to have the whole megabyte that's available.

So a malloc call can end up increasing the physical memory available to
a process - and it will just grab the first physical page from the free
list, then map it into the process.

So I have paging. Of a sort.

What I cannot do is write _some_ of those pages out to disc, and then
allow a VSI to bring them in.

(In practice we never wrote any of them to disc; Digital Research's
dispatcher would get upset if setting up the pages for a process took
significant time.)

The machine I have here has 5MB of RAM.

I could run 8 processes on it, each taking half a meg, and all would be
well. Each process has "an address space" of a megabyte, and would be
happy thinking it could go ahead and malloc to fill up its entire megabyte.

That would be true of one of them. But only one.

Because we don't have virtual memory.

(actually I have a vague feeling there was a way of sharing pages
between processes, but it's a long time ago...)

It's kind of scary that I started out with mainframes where a cabinet
8ft tall and 3ft square was needed for a megabyte of RAM.

There's a device sitting on my desk the size of my thumb with a gigabyte
in it.

(It's an ARM based device BTW, and runs Linux, and everything is there
for virtual memory. But we don't because it would wreck the flash storage)
Post by Johnny Billquist
Post by Vir Campestris
Post by Johnny Billquist
But yeah, I think pretty much noone every used the 286 much, for
these reasons.
And we used it just as a faster 8086.
Yeah. That might have been a popular use. :-D
  Johnny
Peter Flass
2022-03-30 22:26:40 UTC
Permalink
Post by Vir Campestris
Post by Johnny Billquist
Post by Vir Campestris
Post by Johnny Billquist
Post by Vir Campestris
Thanks all for the pointers. I'll have a look when I get time. Which
ought to be in May, when I've retired!
Off-chip memory manager - well, you all know that when the '286 came
out it was an attempt to get a bigger address space on top of the
8086 which was limited to 1MB.
Except Intel did it in such a way almost nobody could use it. OS/2
was about the only one I know of, and that wouldn't really run 8086
programmes.
What we did was to take the top address bits and use them to index
into a fast RAM; the output of the RAM became new address bits.
More address bits.
Which means that within the scope of one process you have the full
1MB address space of the 8086, but the computer can run more than
one process, each with their own address space. You can even do
proper paging if you want. Sadly virtual addressing wasn't possible;
there was no way to restart the current instruction after a page fault.
If you can't restart an instruction, you can't do paging. You'll have
to swap all memory in before execution, so no page faults happen.
Virtual memory is fine though (I think you reversed what paging and
virtual memory is :-) ).
I think we're disagreeing on terminology.
Very likely. But I'll do one attempt at "converting" you. :-)
Post by Vir Campestris
You can have lots of pages on a '286 with the MMU we built. What you
can't have is a page fault. Which means you can't have virtual memory
- everything the app needs must be in real RAM all the time.
Pages basically means the virtual memory is not necessarily all
contiguous in physical memory. I think we can agree on that part. It
also means that you can have a setup where not all of your virtual
memory are mapped somewhere into physical memory at some point in time.
Paging is the action of reading in a page after a page fault. A page
fault happens when a page is not resident in memory.
I do think we can agree on things so far. Possibly you might have some
other definition of what paging is, which would be interesting to hear,
since you think you can do paging.
But the usual definition of paging is that you read/write individual
pages as needed. Which is different than swapping, which is dealing with
all the process memory in one go. With swapping you are either in
memory, or not. You swap a whole process in or out. You don't swap
individual pages. Then we're back to paging again. But even when you
have pages, swapping as such can take place. Nothing says that swapping
cannot be done on an address space that is divided into pages.
Now, virtual memory is a completely different thing. Virtual memory is,
as the name suggest, memory which is virtual, ie. not real/physical. And
it's not real in the sense that the address space your program have
might have an address 0, and you might have some data there. Another
process might also have an address 0, and some data there. But this is
not the same address 0, and it's not the same data. Both processes
thinks they have a bunch of memory which is their own, and it acts like
memory, and can be used. But in the actual machine, there is just
physical memory, and there is an address 0, but it's not the address, or
the data either of these processes are thinking about, referring to, or
using. Each process have its own virtual memory. From time to time, that
virtual memory might map to physical memory, but that might not always
be the case. Further, the actual physical memory address used for
virtual 0 might be anything. And for each process, virtual 0 might map
to a different physical address.
But the point is, from the process point of view, it appears that he has
a full address space with memory, and it's his alone, he can do anything
in there, and it's completely isolated, and the address space as such is
private. He might not even see the OS memory, even though it obviously
also do exist.
The size of physical memory compared to virtual memory is actually
irrelevant. However, if you want to have more virtual memory than there
is physical memory, not only do you need virtual memory, but you also
need paging. Because without paging, you cannot exceed physical memory.
But you can still have virtual memory.
In you head, I bet you are thinking that virtual memory means paging.
But as you observed yourself, you cannot even have page faults if you
can't restart instructions, so at that point your process cannot have
more virtual memory than physical memory. Does that then mean you
actually do not have virtual memory either. You obviously do not have
page faults. What does "paging" mean for you then?
I can have a bunch of discontinuous pages in memory which appear to be
contiguous to the process running.
I can add individual pages to a process as it requests. A small process
doesn't have to have the whole megabyte that's available.
So a malloc call can end up increasing the physical memory available to
a process - and it will just grab the first physical page from the free
list, then map it into the process.
So I have paging. Of a sort.
What I cannot do is write _some_ of those pages out to disc, and then
allow a VSI to bring them in.
(In practice we never wrote any of them to disc; Digital Research's
dispatcher would get upset if setting up the pages for a process took
significant time.)
The machine I have here has 5MB of RAM.
I could run 8 processes on it, each taking half a meg, and all would be
well. Each process has "an address space" of a megabyte, and would be
happy thinking it could go ahead and malloc to fill up its entire megabyte.
That would be true of one of them. But only one.
Because we don't have virtual memory.
(actually I have a vague feeling there was a way of sharing pages
between processes, but it's a long time ago...)
It's kind of scary that I started out with mainframes where a cabinet
8ft tall and 3ft square was needed for a megabyte of RAM.
When I started out, a megabyte was a large memory. Bigger systems had 256K
or 512K.
Post by Vir Campestris
There's a device sitting on my desk the size of my thumb with a gigabyte
in it.
(It's an ARM based device BTW, and runs Linux, and everything is there
for virtual memory. But we don't because it would wreck the flash storage)
Post by Johnny Billquist
Post by Vir Campestris
Post by Johnny Billquist
But yeah, I think pretty much noone every used the 286 much, for
these reasons.
And we used it just as a faster 8086.
Yeah. That might have been a popular use. :-D
  Johnny
--
Pete
J. Clarke
2022-03-31 20:04:05 UTC
Permalink
On Wed, 30 Mar 2022 15:26:40 -0700, Peter Flass
Post by Peter Flass
Post by Vir Campestris
Post by Johnny Billquist
Post by Vir Campestris
Post by Johnny Billquist
Post by Vir Campestris
Thanks all for the pointers. I'll have a look when I get time. Which
ought to be in May, when I've retired!
Off-chip memory manager - well, you all know that when the '286 came
out it was an attempt to get a bigger address space on top of the
8086 which was limited to 1MB.
Except Intel did it in such a way almost nobody could use it. OS/2
was about the only one I know of, and that wouldn't really run 8086
programmes.
What we did was to take the top address bits and use them to index
into a fast RAM; the output of the RAM became new address bits.
More address bits.
Which means that within the scope of one process you have the full
1MB address space of the 8086, but the computer can run more than
one process, each with their own address space. You can even do
proper paging if you want. Sadly virtual addressing wasn't possible;
there was no way to restart the current instruction after a page fault.
If you can't restart an instruction, you can't do paging. You'll have
to swap all memory in before execution, so no page faults happen.
Virtual memory is fine though (I think you reversed what paging and
virtual memory is :-) ).
I think we're disagreeing on terminology.
Very likely. But I'll do one attempt at "converting" you. :-)
Post by Vir Campestris
You can have lots of pages on a '286 with the MMU we built. What you
can't have is a page fault. Which means you can't have virtual memory
- everything the app needs must be in real RAM all the time.
Pages basically means the virtual memory is not necessarily all
contiguous in physical memory. I think we can agree on that part. It
also means that you can have a setup where not all of your virtual
memory are mapped somewhere into physical memory at some point in time.
Paging is the action of reading in a page after a page fault. A page
fault happens when a page is not resident in memory.
I do think we can agree on things so far. Possibly you might have some
other definition of what paging is, which would be interesting to hear,
since you think you can do paging.
But the usual definition of paging is that you read/write individual
pages as needed. Which is different than swapping, which is dealing with
all the process memory in one go. With swapping you are either in
memory, or not. You swap a whole process in or out. You don't swap
individual pages. Then we're back to paging again. But even when you
have pages, swapping as such can take place. Nothing says that swapping
cannot be done on an address space that is divided into pages.
Now, virtual memory is a completely different thing. Virtual memory is,
as the name suggest, memory which is virtual, ie. not real/physical. And
it's not real in the sense that the address space your program have
might have an address 0, and you might have some data there. Another
process might also have an address 0, and some data there. But this is
not the same address 0, and it's not the same data. Both processes
thinks they have a bunch of memory which is their own, and it acts like
memory, and can be used. But in the actual machine, there is just
physical memory, and there is an address 0, but it's not the address, or
the data either of these processes are thinking about, referring to, or
using. Each process have its own virtual memory. From time to time, that
virtual memory might map to physical memory, but that might not always
be the case. Further, the actual physical memory address used for
virtual 0 might be anything. And for each process, virtual 0 might map
to a different physical address.
But the point is, from the process point of view, it appears that he has
a full address space with memory, and it's his alone, he can do anything
in there, and it's completely isolated, and the address space as such is
private. He might not even see the OS memory, even though it obviously
also do exist.
The size of physical memory compared to virtual memory is actually
irrelevant. However, if you want to have more virtual memory than there
is physical memory, not only do you need virtual memory, but you also
need paging. Because without paging, you cannot exceed physical memory.
But you can still have virtual memory.
In you head, I bet you are thinking that virtual memory means paging.
But as you observed yourself, you cannot even have page faults if you
can't restart instructions, so at that point your process cannot have
more virtual memory than physical memory. Does that then mean you
actually do not have virtual memory either. You obviously do not have
page faults. What does "paging" mean for you then?
I can have a bunch of discontinuous pages in memory which appear to be
contiguous to the process running.
I can add individual pages to a process as it requests. A small process
doesn't have to have the whole megabyte that's available.
So a malloc call can end up increasing the physical memory available to
a process - and it will just grab the first physical page from the free
list, then map it into the process.
So I have paging. Of a sort.
What I cannot do is write _some_ of those pages out to disc, and then
allow a VSI to bring them in.
(In practice we never wrote any of them to disc; Digital Research's
dispatcher would get upset if setting up the pages for a process took
significant time.)
The machine I have here has 5MB of RAM.
I could run 8 processes on it, each taking half a meg, and all would be
well. Each process has "an address space" of a megabyte, and would be
happy thinking it could go ahead and malloc to fill up its entire megabyte.
That would be true of one of them. But only one.
Because we don't have virtual memory.
(actually I have a vague feeling there was a way of sharing pages
between processes, but it's a long time ago...)
It's kind of scary that I started out with mainframes where a cabinet
8ft tall and 3ft square was needed for a megabyte of RAM.
When I started out, a megabyte was a large memory. Bigger systems had 256K
or 512K.
I remember when someone described the large data center where he
worked as a "2 gigabyte shop". My phone has a terabyte (I don't
_need_ a terabyte of storage in my phone, but it cracks me up every
time I think about it).
Post by Peter Flass
Post by Vir Campestris
There's a device sitting on my desk the size of my thumb with a gigabyte
in it.
(It's an ARM based device BTW, and runs Linux, and everything is there
for virtual memory. But we don't because it would wreck the flash storage)
Post by Johnny Billquist
Post by Vir Campestris
Post by Johnny Billquist
But yeah, I think pretty much noone every used the 286 much, for
these reasons.
And we used it just as a faster 8086.
Yeah. That might have been a popular use. :-D
  Johnny
Charlie Gibbs
2022-04-01 00:08:30 UTC
Permalink
Post by J. Clarke
On Wed, 30 Mar 2022 15:26:40 -0700, Peter Flass
Post by Peter Flass
Post by Vir Campestris
It's kind of scary that I started out with mainframes where a cabinet
8ft tall and 3ft square was needed for a megabyte of RAM.
The core boxes on the 360/67 at university were that big, but each one
only stored 256K.
Post by J. Clarke
Post by Peter Flass
When I started out, a megabyte was a large memory. Bigger systems
had 256K or 512K.
I worked on smaller mainframes, e.g. Univac 90/30. You could hang
256K on one, but most users settled for 192K to save money.

A predecessor, the 9400, had cabinets half the size of one of those
8x3x3 boxes mentioned above; each held two 64K modules.
Post by J. Clarke
I remember when someone described the large data center where he
worked as a "2 gigabyte shop". My phone has a terabyte (I don't
_need_ a terabyte of storage in my phone, but it cracks me up every
time I think about it).
That 360/67 had two 2314 subsystems. Call it 400MB.
Post by J. Clarke
Post by Peter Flass
Post by Vir Campestris
There's a device sitting on my desk the size of my thumb with a gigabyte
in it.
The one in my pocket holds 16GB; newer ones hold much more.

I remember in my first job when we expanded our machine from 16K to 32K.
We wondered what we were going to do with all that space...

Is it time to start the Four Yorkshiremen skit yet?
--
/~\ Charlie Gibbs | Microsoft is a dictatorship.
\ / <***@kltpzyxm.invalid> | Apple is a cult.
X I'm really at ac.dekanfrus | Linux is anarchy.
/ \ if you read it the right way. | Pick your poison.
J. Clarke
2022-04-01 05:30:42 UTC
Permalink
On Fri, 01 Apr 2022 00:08:30 GMT, Charlie Gibbs
Post by Charlie Gibbs
Post by J. Clarke
On Wed, 30 Mar 2022 15:26:40 -0700, Peter Flass
Post by Peter Flass
Post by Vir Campestris
It's kind of scary that I started out with mainframes where a cabinet
8ft tall and 3ft square was needed for a megabyte of RAM.
The core boxes on the 360/67 at university were that big, but each one
only stored 256K.
Post by J. Clarke
Post by Peter Flass
When I started out, a megabyte was a large memory. Bigger systems
had 256K or 512K.
I worked on smaller mainframes, e.g. Univac 90/30. You could hang
256K on one, but most users settled for 192K to save money.
A predecessor, the 9400, had cabinets half the size of one of those
8x3x3 boxes mentioned above; each held two 64K modules.
Post by J. Clarke
I remember when someone described the large data center where he
worked as a "2 gigabyte shop". My phone has a terabyte (I don't
_need_ a terabyte of storage in my phone, but it cracks me up every
time I think about it).
That 360/67 had two 2314 subsystems. Call it 400MB.
Post by J. Clarke
Post by Peter Flass
Post by Vir Campestris
There's a device sitting on my desk the size of my thumb with a gigabyte
in it.
The one in my pocket holds 16GB; newer ones hold much more.
I remember in my first job when we expanded our machine from 16K to 32K.
We wondered what we were going to do with all that space...
Who was it that said "the data to be stored will expand to fill the
storage available"?
Post by Charlie Gibbs
Is it time to start the Four Yorkshiremen skit yet?
Bob Eager
2022-04-01 10:30:16 UTC
Permalink
Post by J. Clarke
Who was it that said "the data to be stored will expand to fill the
storage available"?
Whoever it was, it was just a corollary of Cyril Northcote Parkinson's
"Parkinson's Law", which was:

"work expands so as to fill the time available for its completion."

And that was in 1955.

Cyril was a nice man. I knew him, and occasionally even had dinner with
him.
--
Using UNIX since v6 (1975)...

Use the BIG mirror service in the UK:
http://www.mirrorservice.org
Johnny Billquist
2022-04-02 17:57:29 UTC
Permalink
Post by Vir Campestris
Post by Johnny Billquist
Post by Vir Campestris
I think we're disagreeing on terminology.
Very likely. But I'll do one attempt at "converting" you. :-)
Post by Vir Campestris
You can have lots of pages on a '286 with the MMU we built. What you
can't have is a page fault. Which means you can't have virtual memory
- everything the app needs must be in real RAM all the time.
Pages basically means the virtual memory is not necessarily all
contiguous in physical memory. I think we can agree on that part. It
also means that you can have a setup where not all of your virtual
memory are mapped somewhere into physical memory at some point in time.
Paging is the action of reading in a page after a page fault. A page
fault happens when a page is not resident in memory.
I do think we can agree on things so far. Possibly you might have some
other definition of what paging is, which would be interesting to
hear, since you think you can do paging.
But the usual definition of paging is that you read/write individual
pages as needed. Which is different than swapping, which is dealing
with all the process memory in one go. With swapping you are either in
memory, or not. You swap a whole process in or out. You don't swap
individual pages. Then we're back to paging again. But even when you
have pages, swapping as such can take place. Nothing says that
swapping cannot be done on an address space that is divided into pages.
Now, virtual memory is a completely different thing. Virtual memory
is, as the name suggest, memory which is virtual, ie. not
real/physical. And it's not real in the sense that the address space
your program have might have an address 0, and you might have some
data there. Another process might also have an address 0, and some
data there. But this is not the same address 0, and it's not the same
data. Both processes thinks they have a bunch of memory which is their
own, and it acts like memory, and can be used. But in the actual
machine, there is just physical memory, and there is an address 0, but
it's not the address, or the data either of these processes are
thinking about, referring to, or using. Each process have its own
virtual memory. From time to time, that virtual memory might map to
physical memory, but that might not always be the case. Further, the
actual physical memory address used for virtual 0 might be anything.
And for each process, virtual 0 might map to a different physical
address.
But the point is, from the process point of view, it appears that he
has a full address space with memory, and it's his alone, he can do
anything in there, and it's completely isolated, and the address space
as such is private. He might not even see the OS memory, even though
it obviously also do exist.
The size of physical memory compared to virtual memory is actually
irrelevant. However, if you want to have more virtual memory than
there is physical memory, not only do you need virtual memory, but you
also need paging. Because without paging, you cannot exceed physical
memory. But you can still have virtual memory.
In you head, I bet you are thinking that virtual memory means paging.
But as you observed yourself, you cannot even have page faults if you
can't restart instructions, so at that point your process cannot have
more virtual memory than physical memory. Does that then mean you
actually do not have virtual memory either. You obviously do not have
page faults. What does "paging" mean for you then?
I can have a bunch of discontinuous pages in memory which appear to be
contiguous to the process running.
Yes. And not only that, the physical addresses of those pages have no
real relationship to the virtual addresses they are being referred to in
the process. It's all just recorded in the translation table.
Post by Vir Campestris
I can add individual pages to a process as it requests. A small process
doesn't have to have the whole megabyte that's available.
Well. You don't actually request pages. You request memory, which might
mean pages are added.

But this is equally true and possible with a system that do not have
pages or paging at all.

Again, take the PDP-11 for example. 2.11BSD, which is a Unix system
running on that hardware. You do not deal with pages. But you also do
not necessarily have all 64K of address space actually mapping to
physical memory. And if you request more memory, your memory space will
be expanded so that you have more memory. This might require some
swapping, in order to place you somewhere in memory where this new,
larger space can be accommodated. No pages were hurt in the process. :-)
Post by Vir Campestris
So a malloc call can end up increasing the physical memory available to
a process - and it will just grab the first physical page from the free
list, then map it into the process.
So I have paging. Of a sort.
Fair enough. The expansion of memory means you might add pages to your
working set. Which do not require any page fault. But there isn't any
actual paging done here. Nothing is read or written to any backing
store. All you do is find some free memory and add it to your set.

But this usecase can also be solved, as mentioned above, without even
have any page implementation at all.
And you would not be able to tell the difference.
Post by Vir Campestris
What I cannot do is write _some_ of those pages out to disc, and then
allow a VSI to bring them in.
Right. Because that would require the restartability of the instruction.
But the actual bringing in and out of pages is what is usually what the
term "paging" refers to.
Post by Vir Campestris
(In practice we never wrote any of them to disc; Digital Research's
dispatcher would get upset if setting up the pages for a process took
significant time.)
The machine I have here has 5MB of RAM.
My PDP-11 have 4M. Not that different.
Post by Vir Campestris
I could run 8 processes on it, each taking half a meg, and all would be
well. Each process has "an address space" of a megabyte, and would be
happy thinking it could go ahead and malloc to fill up its entire megabyte.
Yes.
Post by Vir Campestris
That would be true of one of them. But only one.
Depending on the number of processes you have, you'll hit that limit
sooner or later. It's basically just a question of the system running
out of hardware resources to fulfill the demand.

That happens even today on any modern machine. If all processes try to
allocate as much memory as they could in theory address, the system will
run out of resources, and deny the requests.
Post by Vir Campestris
Because we don't have virtual memory.
No. Because you don't have more physical memory. :-)
Post by Vir Campestris
(actually I have a vague feeling there was a way of sharing pages
between processes, but it's a long time ago...)
It's kind of scary that I started out with mainframes where a cabinet
8ft tall and 3ft square was needed for a megabyte of RAM.
There's a device sitting on my desk the size of my thumb with a gigabyte
in it.
Yeah. Crazy evolution.
Post by Vir Campestris
(It's an ARM based device BTW, and runs Linux, and everything is there
for virtual memory. But we don't because it would wreck the flash storage)
Yeah. Paging isn't good for flash storage. But you still have virtual
memory. It's just that you don't have any backing storage for the
memory, so you have lower limits on memory.

Fun fact - even without any swap space, you will be able to use more
virtual memory than you have physical memory, if you can just restart
instructions (which ARM can do). Basically, when you run a program, any
unmodified pages, which comes from the binary itself, do not need any
other backing store, but the pages can be out. If needed, they are just
read in from the binary again. In fact, when you start execution of a
program, it does not read all of the binary into memory in the first
place. Why should it? It will take a lot of time (the binary might be
big), and it might even be that some parts of it is never used. Starting
a new program basically means you start with no physical memory setup,
and the first thing that happens is that the execution of the first
instruction in the program causes a page fault, and the page is read in,
and then execution is restarted again, and now it will get a bit further
before the next page fault. But after a short while, the most relevant
parts of the program will be in physical memory, and things will be running.

Johnny

Scott Lurndal
2022-03-21 23:15:09 UTC
Permalink
Post by Vir Campestris
This is something I've thought about for a while.
I have an old computer here which runs a '286 in real address mode with
an off-chip memory manager. I have fond memories of the project, and
would like to get an emulator of it running.
But I'm not about to write a '286 emulator from the ground up. I tried
it once for an 8080, and that was painful enough - it never worked
properly even though I could just about get CP/M to run DDT.
qemu will emulate the '286.

https://www.qemu.org/docs/master/system/target-i386.html
Peter Maydell
2022-03-27 22:23:36 UTC
Permalink
Post by Scott Lurndal
Post by Vir Campestris
This is something I've thought about for a while.
I have an old computer here which runs a '286 in real address mode with
an off-chip memory manager. I have fond memories of the project, and
would like to get an emulator of it running.
But I'm not about to write a '286 emulator from the ground up. I tried
it once for an 8080, and that was painful enough - it never worked
properly even though I could just about get CP/M to run DDT.
qemu will emulate the '286.
https://www.qemu.org/docs/master/system/target-i386.html
No, the oldest x86 CPU type QEMU has support for is the 486.

TBH, I wouldn't really recommend QEMU for this kind of retrocomputing
emulation task anyway. QEMU is more focused on running guest code
quickly and without worrying about anything like timing accuracy.
That works fine for running Linux and other comparatively modern code
which has been written to cope with a wide range of CPU speed and
timing anyway, and you need the performance. But older systems more
often had software that accidentally or deliberately relied on timing
specifics of the hardware, and were so slow that you can get
better-than-hardware performance on a modern PC even using slow
timing-accurate emulation approaches. Plus QEMU is not an easy
emulator to add new machine types to (largely because the APIs
and design approaches to do so are not documented and you have
to spend a lot of time reading source code of existing models).

I've never worked with MAME, but it would be high on my shortlist
of possibilities to investigate if I were doing this.

-- PMM
Vir Campestris
2022-03-28 20:44:39 UTC
Permalink
Post by Peter Maydell
No, the oldest x86 CPU type QEMU has support for is the 486.
TBH, I wouldn't really recommend QEMU for this kind of retrocomputing
emulation task anyway. QEMU is more focused on running guest code
quickly and without worrying about anything like timing accuracy.
That works fine for running Linux and other comparatively modern code
which has been written to cope with a wide range of CPU speed and
timing anyway, and you need the performance. But older systems more
often had software that accidentally or deliberately relied on timing
specifics of the hardware, and were so slow that you can get
better-than-hardware performance on a modern PC even using slow
timing-accurate emulation approaches. Plus QEMU is not an easy
emulator to add new machine types to (largely because the APIs
and design approaches to do so are not documented and you have
to spend a lot of time reading source code of existing models).
I've never worked with MAME, but it would be high on my shortlist
of possibilities to investigate if I were doing this.
MAME doesn't sound right for me.

This is a retirement project (I put away my card punch next month) and a
'486 in real mode should do the job.

I _know_ I would have trouble with IO delay loops - if, that is, I was
driving real hardware. But I won't be, so I won't have to allow for IO
recovery times on slow devices. Emulating a SCSI disc on a SCSI
controller should be the most painful bit!

Thanks

Andy
Peter Maydell
2022-03-28 21:46:45 UTC
Permalink
Post by Vir Campestris
Post by Peter Maydell
No, the oldest x86 CPU type QEMU has support for is the 486.
TBH, I wouldn't really recommend QEMU for this kind of retrocomputing
emulation task anyway. QEMU is more focused on running guest code
quickly and without worrying about anything like timing accuracy.
That works fine for running Linux and other comparatively modern code
which has been written to cope with a wide range of CPU speed and
timing anyway, and you need the performance. But older systems more
often had software that accidentally or deliberately relied on timing
specifics of the hardware, and were so slow that you can get
better-than-hardware performance on a modern PC even using slow
timing-accurate emulation approaches. Plus QEMU is not an easy
emulator to add new machine types to (largely because the APIs
and design approaches to do so are not documented and you have
to spend a lot of time reading source code of existing models).
I've never worked with MAME, but it would be high on my shortlist
of possibilities to investigate if I were doing this.
MAME doesn't sound right for me.
This is a retirement project (I put away my card punch next month) and a
'486 in real mode should do the job.
Well, as I say, I've never worked with MAME. But even if a 486 is
close enough I dunno that I'd care to try to implement the ad-hoc
pager you describe with QEMU (the only approaches I can think of
are either really ugly or else hacking up the bit of the code
that does the x64 mmu emulation to do something else instead).
Something that does a better job of providing flexible building
blocks to put together into an emulated system seems like it
would be a better fit.

-- PMM
Acceptable Name
2022-03-28 16:06:20 UTC
Permalink
Post by Vir Campestris
This is something I've thought about for a while.
I have an old computer here which runs a '286 in real address mode with
an off-chip memory manager. I have fond memories of the project, and
would like to get an emulator of it running.
But I'm not about to write a '286 emulator from the ground up. I tried
it once for an 8080, and that was painful enough - it never worked
properly even though I could just about get CP/M to run DDT.
Is there a decent open source '286 emulator, ideally written in C++
although C would do fine, that I could use to build my emulator around?
Andy
These are the two I know of that nobody mentioned here yet.

PCem is still maintained despite rumors of its death.
http://pcem-emulator.co.uk/
https://github.com/sarah-walker-pcem/pcem
https://github.com/BaRRaKudaRain/PCem-ROMs

86Box
https://86box.net/
https://github.com/86Box
https://86box.readthedocs.io/en/latest/index.html
Loading...