Update – PiDP8 and text files

Looking back on my previous posts about the PiDP8 – a PDP8 replica that uses a Raspberry Pi as it’s compute engine, I realized that I never explained how I got text files (i.e. FORTRAN source and data files) from my Raspberry Pi into the PiDP8 emulator.

The process involves several steps. In overview, they are

  • copy the file to a directory on the Pi where SIMH can find it.
  • from within the PiDP8, escape to SIMH (<ctrl-e>)
  • in SIMH, attach the file to a Paper Tape Reader (a device supported by SIMH)
  • CON to return to the PiDP8
  • copy the file from paper tape reader to the PiDP8 disk using PIP

These steps are not all that difficult once you’ve done them a few times.

SIMH can only look for files in the SIMH execution path. That includes imagefiles (where it finds disk and tape images), so I used that as a starting location. I created a new directory /opt/pidp8/imagefiles/rsh to contain any text files I want to transfer to the PiDP8. To make things easier, in the home directory, I created a directory to hold the text files when I transfer them from my PC. I then created a symbolic link in that directory to the /opt/pidp8/imagefiles/rsh so it’s easy to copy files there.

From within the PiDP8, <ctrl-e> transfers control to SIMH. In SIMH, attaching a text file to the paper tape reader (PTR) is done wiht the command ‘att ptr ../imagefiles/rsh/text_file’. The text file can have any valid unix name. However, once you copy it to the PiDP8, you need to use PDP8 file naming conventions (6 char max + 2 char extension). Typical extensions are .FT for a  fortran source, .DA for a data file, and .TX for a text file. Once the file is attached to PTR, return to PiDP8 with CON.

In PiDP8, I found the best program to read the paper tape reader and store a file is PIP. Sadly, PIP is not on the default boot disk image. If you refer back to my post “Getting FORTRAN on the PiDP8”, that technique can be used to boot the image ‘os8.tu56’.  It needs to be attached to td0 (tape 0) and then you boot td0 and copy the file to RKA0 before rebooting rk0.

PIP is easy to use, with one note. If anything goes wrong with the copy operation, you must go back to SIMH and detach, then reattach the text file to PTR. To copy the file from PTR to disk, the command ‘R PIP’ is used. It return an asterisk meaning ‘command in progress’. Type ‘FILE.TX < PTR: /I’ to copy the file from PTR to FILE.TX on the PiDP8 disk (RKA0 by default). You can use ‘RKB0:FILE.TX < PTR: /I’ to copy it to RKB0 if you wish instead. While copying, you get a ‘^’  symbol, then the ‘*’ prompt again. Press “esc” and “return” to complete the copy.

Now you can go back to SIMH and DET PTR to detach the file from PTR if you wish.

Cranky tools make for cranky developers

I’ve been compiling and testing the Open Porous Media (OPM) reservoir simulators for over a week now, and have discovered one very annoying ‘feature’ of the Ubuntu package tool.

I have had great success compiling OPM under Ubuntu 14.04, which is the ‘official’ version for OPM at this time.

I have had less than great success compiling OPM under Ubuntu 16.04 for the past week. Once I did a lot of manual tweaking and got it to compile, but every time since I’ve had the build fail. It isn’t easy to diagnose as the log files run to thousand of lines.

Tonight I figured out the problem.,, and it’s made me a bit cranky.

The issue is that there are package differences between Ubuntu 14 and 16, but I already knew that. It’s what the package manager does with some differences that’s annoying.

The command to install a package in Ubuntu is ‘apt-get install’, followed by one or more packages. So far, so good. But here’s the problem… if you specify multiple packages in the list, and any one of them no longer exists, the ENTIRE INSTALL fails. Worse, there’s no real error message. You only figure it out by the lack of messages about the packages that didn’t get installed.

For example, say you run ‘sudo apt-get install packa packb packd1.02 packe

If packd1.02 existed in U14 but now it’s packd1.11 (for example), you do get a message in the log telling you that packd1.02 cannot be found and thus cannot be installed. What you don’t get is any message telling you that it’s aborted the entire install and so packa, packb and packe are also NOT installed.

It took a few days to track down (why was BOOST missing all the time???) but finally I cracked the code, and it looks like the big compile is proceeding as expected.

It figures, Microsoft

I tend to keep everything. There’s a long story behind that going back to my dad and our move from Calgary to Red Deer when I was 10, but I’ll leave that for another time.

I’m not as much of a ‘physical stuff’ pack-rat as I used to be; moving a long distance cures a lot of that. But I am a digital pack-rat, and have been since my first experiences with computers. As a result, I have various documents I’ve written dating back to my undergraduate days. Many things were saved as text, but once word processors came about I started to save their documents as well. Other than one nasty episode where I’ve lost my entire M.Eng. Thesis because it was saved in Borland’s orphaned (and forever forgotten) “Sprint” editor, my stuff is mostly Microsoft Word documents.

After all, MS Word has been around since early Windows, and it’s still sold and maintained. What could possibly be wrong with saving old MS Word documents, correct?

Well, as it turns out, plenty. MS Word has dropped support for older products as time passed. Each new version seems to lose support for another older version. As a result, when I went to look at a 1992 MS Word document last night, it would not open. There were a few, and some would open but not display properly,  but some would not open at all. The error was “check the file protection settings in trust center”. I did that, but I already had as much prior support enabled as possible.

I even tried my older copy of MS Word on my 2005 Macbook. No joy there either.

I figured that the files were lost to time, thanks to Microsoft’s laziness and lack of care.

But.. this past week I’ve been playing with Open Source programs, most notably the FLOW reservoir simulator. I was compiling that from source on an Ubuntu 14.04 virtual machine running on VirtualBox on my Win7 PC with great success and fun. I also had created a VirtualBox Ubuntu 16.04 machine for compiling Tomcat and other Apache programs.

I remembered that all the Ubuntu machines were installed by default with a current copy of LibraOffice – one of the open source word processors.

On a whim, I fired up Ubuntu 16.04 on VirtualBox, used SMB to connect to my data server, and copied a directory of the older MS Word documents over to a local directory on Ubuntu. *** Programmers Rule #1: NEVER work on the original file.***

Once the documents were copied, I double-clicked the one that would not open at all in new MS Word, and … IT OPENED! No errors, not missing formatting. It was there, perfectly intact.

So, way to go Microsoft. You ditched compatibility with your own older document formats, but fortunately the Open Source world didn’t.

One more  win for Open Source.

Reservoir Simulation Revisited

Back in the early 1980’s, I worked for some years in Calgary writing reservoir simulators in FORTRAN for one company, and a few more years serving as technical support for another company. I really enjoyed the work and the intellectual stimulation of doing reservoir simulation.

Then my career moved to more traditional IT consulting work before I ended up working in education. I started teaching programming as a continuing education instructor, then became a daytime contract instructor before working full-time as an instructor with Athabasca University.

Over the years I’ve played around with modern reservoir simulators, even going as far to obtain an academic license for a commercial simulator from a company run by the son of one of the senior simulation researchers I had worked with years before.

When I started playing with FORTRAN again on my PiDP8 replica, I started to get interested my old Engineering work, going as far as to convert, compile and run all my old Engineering graduate studies programs.

But I wanted to play with reservoir simulation again.

So I started looking for open source reservoir simulators, and I found an excellent one in the Open Porous Media (OPM) project. They offer reservoir simulators and ancillary programs, all under the open license. Best of all, if you dig around a bit on the OPM website, you discover that it’s possible to compile everything from  source.

I have now obtained, built and installed the entire suite of simulation programs that OPM currently has, including FLOW (the main reservoir simulator) and ResInsight (a data visualizer). I have created repeatable scripts and documentation that allows me to build everything on a VirtualBox machine running Ubuntu 14.04, all running on my Windows7 PC. Ubuntu 14.04 is not the latest (that’s 16.04 currently), but that’s the version they recommend in the OPM wiki files.

It executes the sample simulations quite quckly, which is nice. They also have as examples some of the old  SPE Comparative Solution Project test cases that I used to run in the 1980s. It’s turning out to be really great fun.

My next steps are to expand the simulations I run beyond the simple test cases, and then explore some of the source code. Perhaps I can even contribute something to the project in time.

Virtualbox & Ubuntu

I now have three Ubuntu VM’s (virtual machines) on my Win7 PC running under VirtualBox.

Machine One is the ‘Firefox Build Environment’ that uses Ubuntu 16 and comes with Sublime-Text preinstalled with a host of macros to get the latest Firefox source and build the program. It works and was really fun to use. It’s also a bit boring as it’s kind-of single purpose for me. I really don’t want to mess with it as all this “stuff” is pre-configured and works.

So I obtained Ubuntu 16.04 and installed it as Machine Two. After installing it, I wrote a document about what I did. I then decided to try something radical (for me): I obtained Tomcat from source and compiled it. This required getting packages (apt-get) for Subversion, Java and other tools and libraries that Tomcat required. I was then able to get the source using SVN (subversion) and compile it following the Tomcat build documents. In the end it turned out to be very easy and fun, plus now I have Tomcat 9 (latest) running perfectly on Machine Two.

Now for the tough one. I started looking for an Open Source Reservoir Simulator, and found FLOW, from the OPM (Open Porous Media Project found at http://opm-project.org/. It was exactly what I was looking for: both Open Source and something I might be able to really get into again.

Although binaries are available, I really wanted to install from source, just for the fun of doing it all. After much reading, I obtained and installed Ubuntu 14.04 for Machine Three. This was the last version where OPM said everything compiled without problems.

After even more reading, I was able to obtain the sources and create shell scripts to download, configure and compile FLOW from source. I was also able to run the SPE test cases from my early days in industry successfully. Again, it was rather fun.

I also have other machines now – Fedora, OpenBSD, Debian and SAS, but the OPM machine is my favorite right now. I’ll write more on FLOW in another post.

VirtualBox – now on my Win7 machine

I have had VirtualBox running on my Athabascau University issued Macbook Pro for several months now. It’s a bit of a story…

I started down the road to virtualization on the Mac with Parallels. It worked well and wasn’t too costly… at first). Eventually I dumped Parallels as they kept wanting $69.99 more and more frequently for upgrades that didn’t really seem to offer much. Sadly for them, the last cash request came at the same time VMWare sent an offer to switch at a price that was too good to refuse. I dumped them for VMWare and that kept me happy for a few years.

However, VMWare also fell prey to the “cash for upgrade” mania and started doing the same thing. That’s when I searched for a free alternative and found VirtualBox.

Now VirtualBox has it’s problems. The biggest one is that it’s yet another Oracle product (like Java and Solaris and MySQL). Still, I use the three named products and so Oracle seems to be OK… for now.

I installed VirtualBox on the Mac and recreated my critical virtual machines – Win 7 (for netbeans marking), Win 98 (for LEGO Mindstorms), OpenBSD (for server testing), Minix 3 (Operating systems playing) and Windows 10 (again, for testing). It works well.

However, my latest tests failed without warning and without real recourse.

I just downloaded the Linux/Firefox development machine for VirtualBox from Mozilla.  It installed and started without problems, but when I went to build Firefox from the latest source as you are instructed to do in the guide, THE BUILD FAILED. Lots of error messages, but no help: the guide said it would be successful.

In an attempt to see if it was the VM or my Mac, I decided to install VirtualBox on my main Windows7 PC. It’s daunting as it warns you it will insert “stuff” into the network stack, and my Win7 network has always been a bit… weird.

Still, I could always uninstall. So I installed VirtualBox today and then gave it a try on the minix3 image (exported from the Mac), the SAS image I have and the Firefox image. BOOM!!!

Well, not boom, but *wimper*. It wouldn’t run. The error message was traced with google help to a BIOS switch that has virtual machines turned off for my system. I had to reboot, <F2> to modify BIOS settings to turn VM on, then save and reboot. However, once done VirtualBox worked.  I did have to re-import the images but then everything worked.

Now the acid test – build Firefox on the Ubuntu VM. After well over an hour, it was SUCCESS. All my networking still seems to work OK, and the VM machines all connect to the network without too much difficulty (DNS lookups seem to take some time but do work).

All in all, I’m pretty happy with VirtualBox. First on the Mac, and now on my Win7 PC.

Final thought: I was really rather excited to use the Ubuntu “FIrefox devel” machine to actually download and build Firefox from the latest source snapshots. I’ve been reading a great book “rebel code” about the whole GNU/Linux scene and have become interested in playing with both Linux and some of the open source programs as a developer rather than just a user. More on the book in another post.