Network Weirdness – Update

As noted in the previous (October) post on my network ‘weireness’, the new keyboard has resolved all issues with the mysterious key presses and characters appearing when connecting to unix boxes of all flavors and locations.

What did not go away was another problem – mysterious session ‘resets’ that would occur randomly during file transfers from my remote co-locate server. Simply put, for no apparent reason and at random but frequent intervals, the download would stop and the remote unix session would die with a “session reset” error.

After observing exactly what was going on when the session would die, I realized that the only common occurrance was that I was using a browser (Netscape) or email (Thunderbird) to access web pages or email. This seemed odd, but deserved a closer look.

I don’t really want to play super detective with the network stack. Wireshark is awesome, but diagnosing what exactly caused the event you capture is tedious and frequently without a solid answer. I could see the session dying, but the reason “unexpected xyz in abc” is not that much help when it’s something in the windows stack that’s involved.

Instead, I tried a simple test: don’t use email or browse the web AT ALL during the download. Just wait. I’ve done this twice now and the downloads proceeded without any interruption and the session never died. A third time I let it go for quite a while, then started browsing and … the session died. Quit browsing and the session ended normally.

Since downloads of backup data can take several tens of minutes, waiting can be boring, but it’s still better than resetting the session frequently.

Network weirdness… resolved?

Back on March 13, 2016, I posted the following:

As for my network speed, it started because I was getting really tired of my SSH sessions being clogged with strange control characters and other weirdness including sudden session drops. When I went to check SSH on other machines including my mac and my other win7 box, there were no issues at all. That meant it was my current win7 machine.

In that post, I indicated that updating the NVIDEA drivers and disabling some of the more useless driver functions helped, but did not cure the problem.

Slow forward to last week. The problem still persisted. It did not matter what machine I connected to with SSH, it was always bad. Worst was the PiDP8. It was totally impossible to use the PDP8 editor because the spurious control characters generated my Win7 ssh sessions always killed the edit. I ended up using other means to get files onto the PiDP8.

It was bad in any SSH session – unix, linux, Raspbian, Ubuntu, OpenBSD. They all suffered. Even newer releases of Putty and other SSH clients did NOT fix the problem.

However…

My lighted Logitech keyboard started to fail. Gradually. Slowly. But eventually the control key was so bad that “ctrl-c + ctrl-v” did not cut-and-paste, but rather just inserted ‘c’ and ‘v’ wherever used. Working with Excel became almost impossible. Emails had to be carefully checked to ensure I had not messed them up.

I finally had enough, and did some Google searches on decent inexpensive lighted keyboards. The only condition was I would not pay over $150 for a keyboard. In the end a high rated keyboard was the AZIO lighted keyboard. $45 with free shipping from Amazon.ca, and wired USB like my old one.

It arrived, I unplugged the old Logitech, plugged in the new Azio… and the machine froze. A quick reboot and it was fine. Probably just a driver hand-over glitch.

But what happened next had me totally baffled.

ALL OF MY SSH SESSIONS WORKED PERFECTLY. Yep – after swapping out the Logitech keyboard for an Azio keyboard, the very next ssh session had no key glitches. Zero. None. Nada. And the session after that, and the session after that, …

It’s now been almost two weeks since the new keyboard, and every ssh login has been clean and glitch free.

My best guess is that either there was something amiss with the logitech keyboard driver, or some weird interaction. The keyboard never gave any problem at all (until the control key failure) when used on windows, ONLY when running an ssh session.

Very strange. Very strange indeed.

Update – PiDP8 and text files

Looking back on my previous posts about the PiDP8 – a PDP8 replica that uses a Raspberry Pi as it’s compute engine, I realized that I never explained how I got text files (i.e. FORTRAN source and data files) from my Raspberry Pi into the PiDP8 emulator.

The process involves several steps. In overview, they are

  • copy the file to a directory on the Pi where SIMH can find it.
  • from within the PiDP8, escape to SIMH (<ctrl-e>)
  • in SIMH, attach the file to a Paper Tape Reader (a device supported by SIMH)
  • CON to return to the PiDP8
  • copy the file from paper tape reader to the PiDP8 disk using PIP

These steps are not all that difficult once you’ve done them a few times.

SIMH can only look for files in the SIMH execution path. That includes imagefiles (where it finds disk and tape images), so I used that as a starting location. I created a new directory /opt/pidp8/imagefiles/rsh to contain any text files I want to transfer to the PiDP8. To make things easier, in the home directory, I created a directory to hold the text files when I transfer them from my PC. I then created a symbolic link in that directory to the /opt/pidp8/imagefiles/rsh so it’s easy to copy files there.

From within the PiDP8, <ctrl-e> transfers control to SIMH. In SIMH, attaching a text file to the paper tape reader (PTR) is done wiht the command ‘att ptr ../imagefiles/rsh/text_file’. The text file can have any valid unix name. However, once you copy it to the PiDP8, you need to use PDP8 file naming conventions (6 char max + 2 char extension). Typical extensions are .FT for a  fortran source, .DA for a data file, and .TX for a text file. Once the file is attached to PTR, return to PiDP8 with CON.

In PiDP8, I found the best program to read the paper tape reader and store a file is PIP. Sadly, PIP is not on the default boot disk image. If you refer back to my post “Getting FORTRAN on the PiDP8”, that technique can be used to boot the image ‘os8.tu56’.  It needs to be attached to td0 (tape 0) and then you boot td0 and copy the file to RKA0 before rebooting rk0.

PIP is easy to use, with one note. If anything goes wrong with the copy operation, you must go back to SIMH and detach, then reattach the text file to PTR. To copy the file from PTR to disk, the command ‘R PIP’ is used. It return an asterisk meaning ‘command in progress’. Type ‘FILE.TX < PTR: /I’ to copy the file from PTR to FILE.TX on the PiDP8 disk (RKA0 by default). You can use ‘RKB0:FILE.TX < PTR: /I’ to copy it to RKB0 if you wish instead. While copying, you get a ‘^’  symbol, then the ‘*’ prompt again. Press “esc” and “return” to complete the copy.

Now you can go back to SIMH and DET PTR to detach the file from PTR if you wish.

Cranky tools make for cranky developers

I’ve been compiling and testing the Open Porous Media (OPM) reservoir simulators for over a week now, and have discovered one very annoying ‘feature’ of the Ubuntu package tool.

I have had great success compiling OPM under Ubuntu 14.04, which is the ‘official’ version for OPM at this time.

I have had less than great success compiling OPM under Ubuntu 16.04 for the past week. Once I did a lot of manual tweaking and got it to compile, but every time since I’ve had the build fail. It isn’t easy to diagnose as the log files run to thousand of lines.

Tonight I figured out the problem.,, and it’s made me a bit cranky.

The issue is that there are package differences between Ubuntu 14 and 16, but I already knew that. It’s what the package manager does with some differences that’s annoying.

The command to install a package in Ubuntu is ‘apt-get install’, followed by one or more packages. So far, so good. But here’s the problem… if you specify multiple packages in the list, and any one of them no longer exists, the ENTIRE INSTALL fails. Worse, there’s no real error message. You only figure it out by the lack of messages about the packages that didn’t get installed.

For example, say you run ‘sudo apt-get install packa packb packd1.02 packe

If packd1.02 existed in U14 but now it’s packd1.11 (for example), you do get a message in the log telling you that packd1.02 cannot be found and thus cannot be installed. What you don’t get is any message telling you that it’s aborted the entire install and so packa, packb and packe are also NOT installed.

It took a few days to track down (why was BOOST missing all the time???) but finally I cracked the code, and it looks like the big compile is proceeding as expected.

It figures, Microsoft

I tend to keep everything. There’s a long story behind that going back to my dad and our move from Calgary to Red Deer when I was 10, but I’ll leave that for another time.

I’m not as much of a ‘physical stuff’ pack-rat as I used to be; moving a long distance cures a lot of that. But I am a digital pack-rat, and have been since my first experiences with computers. As a result, I have various documents I’ve written dating back to my undergraduate days. Many things were saved as text, but once word processors came about I started to save their documents as well. Other than one nasty episode where I’ve lost my entire M.Eng. Thesis because it was saved in Borland’s orphaned (and forever forgotten) “Sprint” editor, my stuff is mostly Microsoft Word documents.

After all, MS Word has been around since early Windows, and it’s still sold and maintained. What could possibly be wrong with saving old MS Word documents, correct?

Well, as it turns out, plenty. MS Word has dropped support for older products as time passed. Each new version seems to lose support for another older version. As a result, when I went to look at a 1992 MS Word document last night, it would not open. There were a few, and some would open but not display properly,  but some would not open at all. The error was “check the file protection settings in trust center”. I did that, but I already had as much prior support enabled as possible.

I even tried my older copy of MS Word on my 2005 Macbook. No joy there either.

I figured that the files were lost to time, thanks to Microsoft’s laziness and lack of care.

But.. this past week I’ve been playing with Open Source programs, most notably the FLOW reservoir simulator. I was compiling that from source on an Ubuntu 14.04 virtual machine running on VirtualBox on my Win7 PC with great success and fun. I also had created a VirtualBox Ubuntu 16.04 machine for compiling Tomcat and other Apache programs.

I remembered that all the Ubuntu machines were installed by default with a current copy of LibraOffice – one of the open source word processors.

On a whim, I fired up Ubuntu 16.04 on VirtualBox, used SMB to connect to my data server, and copied a directory of the older MS Word documents over to a local directory on Ubuntu. *** Programmers Rule #1: NEVER work on the original file.***

Once the documents were copied, I double-clicked the one that would not open at all in new MS Word, and … IT OPENED! No errors, not missing formatting. It was there, perfectly intact.

So, way to go Microsoft. You ditched compatibility with your own older document formats, but fortunately the Open Source world didn’t.

One more  win for Open Source.

Reservoir Simulation Revisited

Back in the early 1980’s, I worked for some years in Calgary writing reservoir simulators in FORTRAN for one company, and a few more years serving as technical support for another company. I really enjoyed the work and the intellectual stimulation of doing reservoir simulation.

Then my career moved to more traditional IT consulting work before I ended up working in education. I started teaching programming as a continuing education instructor, then became a daytime contract instructor before working full-time as an instructor with Athabasca University.

Over the years I’ve played around with modern reservoir simulators, even going as far to obtain an academic license for a commercial simulator from a company run by the son of one of the senior simulation researchers I had worked with years before.

When I started playing with FORTRAN again on my PiDP8 replica, I started to get interested my old Engineering work, going as far as to convert, compile and run all my old Engineering graduate studies programs.

But I wanted to play with reservoir simulation again.

So I started looking for open source reservoir simulators, and I found an excellent one in the Open Porous Media (OPM) project. They offer reservoir simulators and ancillary programs, all under the open license. Best of all, if you dig around a bit on the OPM website, you discover that it’s possible to compile everything from  source.

I have now obtained, built and installed the entire suite of simulation programs that OPM currently has, including FLOW (the main reservoir simulator) and ResInsight (a data visualizer). I have created repeatable scripts and documentation that allows me to build everything on a VirtualBox machine running Ubuntu 14.04, all running on my Windows7 PC. Ubuntu 14.04 is not the latest (that’s 16.04 currently), but that’s the version they recommend in the OPM wiki files.

It executes the sample simulations quite quckly, which is nice. They also have as examples some of the old  SPE Comparative Solution Project test cases that I used to run in the 1980s. It’s turning out to be really great fun.

My next steps are to expand the simulations I run beyond the simple test cases, and then explore some of the source code. Perhaps I can even contribute something to the project in time.

Virtualbox & Ubuntu

I now have three Ubuntu VM’s (virtual machines) on my Win7 PC running under VirtualBox.

Machine One is the ‘Firefox Build Environment’ that uses Ubuntu 16 and comes with Sublime-Text preinstalled with a host of macros to get the latest Firefox source and build the program. It works and was really fun to use. It’s also a bit boring as it’s kind-of single purpose for me. I really don’t want to mess with it as all this “stuff” is pre-configured and works.

So I obtained Ubuntu 16.04 and installed it as Machine Two. After installing it, I wrote a document about what I did. I then decided to try something radical (for me): I obtained Tomcat from source and compiled it. This required getting packages (apt-get) for Subversion, Java and other tools and libraries that Tomcat required. I was then able to get the source using SVN (subversion) and compile it following the Tomcat build documents. In the end it turned out to be very easy and fun, plus now I have Tomcat 9 (latest) running perfectly on Machine Two.

Now for the tough one. I started looking for an Open Source Reservoir Simulator, and found FLOW, from the OPM (Open Porous Media Project found at http://opm-project.org/. It was exactly what I was looking for: both Open Source and something I might be able to really get into again.

Although binaries are available, I really wanted to install from source, just for the fun of doing it all. After much reading, I obtained and installed Ubuntu 14.04 for Machine Three. This was the last version where OPM said everything compiled without problems.

After even more reading, I was able to obtain the sources and create shell scripts to download, configure and compile FLOW from source. I was also able to run the SPE test cases from my early days in industry successfully. Again, it was rather fun.

I also have other machines now – Fedora, OpenBSD, Debian and SAS, but the OPM machine is my favorite right now. I’ll write more on FLOW in another post.

VirtualBox – now on my Win7 machine

I have had VirtualBox running on my Athabascau University issued Macbook Pro for several months now. It’s a bit of a story…

I started down the road to virtualization on the Mac with Parallels. It worked well and wasn’t too costly… at first). Eventually I dumped Parallels as they kept wanting $69.99 more and more frequently for upgrades that didn’t really seem to offer much. Sadly for them, the last cash request came at the same time VMWare sent an offer to switch at a price that was too good to refuse. I dumped them for VMWare and that kept me happy for a few years.

However, VMWare also fell prey to the “cash for upgrade” mania and started doing the same thing. That’s when I searched for a free alternative and found VirtualBox.

Now VirtualBox has it’s problems. The biggest one is that it’s yet another Oracle product (like Java and Solaris and MySQL). Still, I use the three named products and so Oracle seems to be OK… for now.

I installed VirtualBox on the Mac and recreated my critical virtual machines – Win 7 (for netbeans marking), Win 98 (for LEGO Mindstorms), OpenBSD (for server testing), Minix 3 (Operating systems playing) and Windows 10 (again, for testing). It works well.

However, my latest tests failed without warning and without real recourse.

I just downloaded the Linux/Firefox development machine for VirtualBox from Mozilla.  It installed and started without problems, but when I went to build Firefox from the latest source as you are instructed to do in the guide, THE BUILD FAILED. Lots of error messages, but no help: the guide said it would be successful.

In an attempt to see if it was the VM or my Mac, I decided to install VirtualBox on my main Windows7 PC. It’s daunting as it warns you it will insert “stuff” into the network stack, and my Win7 network has always been a bit… weird.

Still, I could always uninstall. So I installed VirtualBox today and then gave it a try on the minix3 image (exported from the Mac), the SAS image I have and the Firefox image. BOOM!!!

Well, not boom, but *wimper*. It wouldn’t run. The error message was traced with google help to a BIOS switch that has virtual machines turned off for my system. I had to reboot, <F2> to modify BIOS settings to turn VM on, then save and reboot. However, once done VirtualBox worked.  I did have to re-import the images but then everything worked.

Now the acid test – build Firefox on the Ubuntu VM. After well over an hour, it was SUCCESS. All my networking still seems to work OK, and the VM machines all connect to the network without too much difficulty (DNS lookups seem to take some time but do work).

All in all, I’m pretty happy with VirtualBox. First on the Mac, and now on my Win7 PC.

Final thought: I was really rather excited to use the Ubuntu “FIrefox devel” machine to actually download and build Firefox from the latest source snapshots. I’ve been reading a great book “rebel code” about the whole GNU/Linux scene and have become interested in playing with both Linux and some of the open source programs as a developer rather than just a user. More on the book in another post.

PiDP8 boot script for MULTOS/8

The PiDP8 has one really neat feature; by setting the first left three white switches, you can control which of 8 boot scripts are used to start the SimH session running the PDP8/I emulation on the Pi.

Oscar provided 8 scripts, from 0.script to 7.script. According to the description file, 5.script is blank. It actually contains assembly instructions for ??? but I just saved that file as 5.script.save, then created a new 5.script which automatically boots the PiDP8 into MULTOS8 maintenance mode if the switches are set to octal 5 = 101.

Here’s my new 5.script:

reset
set cpu 32k
set cpu noidle
att rk0 ../imagefiles/multos8/rsh.rk05
att rk1 ../imagefiles/multos8/multos8.rk05
boot rk1

If you created a running MULTOS/8 system based on my last blog post, you can name the RKA0: disk pack you create to anything you want. I chose rsh.rk05, but any name will do. The .rk05 extension is not mandatory, but appropriate to keep things straight.T800_PiDP8 completed 2016-05-26

MULTOS/8 – a timeshare operating system for the PDP/8

As stated previously in my blog, I’ve been having tons of fun playing with my PDP8/I replica, the PiDP8 designed by Oscar Vermeulen and sold as a kit. The engine for the replica is a Raspberry Pi (3, in my case) running Simh, an awesome ‘old iron’ simulator that  runs on many platforms.

I was intrigued by the idea of MULTOS/8 after reading an inquiry about it on Oscar’s PiDP8 forum. I did a bit of digging, and found that a few folks were interested, but none (apparently) had it running on Simh.

I was able to obtain a copy from the PDP repository at https://www.pdp8online.com/images/images/os8.shtml and began.

The disk image is multos8.rk05, indicating it’s a disk pack much like the standard OS8 operating system. After some playing, I have been able to get MULTOS/8 running successfully on my PiDP8.

The general procedure is to boot OS8, then enter SIMH and attach the multos disk image to RK1, while detaching the OS8 image. In place of the OS8 disk image on RK0, you attach a new file as disk image (mine was called rsh.rk05). Then you boot to MULTOS8 in a maintenance mode. In that mode, you format (ZERO) the blank disk image and copy the MULTOS files to the blank disk. Now you can boot MULTOS into timeshare mode and play.

One quick note on paths for SIMH on the PiDP8:  I put my image files in /opt/pidp8/imagefiles/rsh to keep them separate from Oscar’s default files. SIMH finds files relative to /opt/pidp8, hence the ‘../imagefiles/rsh/rsh.rk05’ path. You are free to put your files where you want as long as SIMH can see them.

Here are the detailed steps:

Begin with the normal OS8 boot.
<ctrl-e> (to get into simh)

*Command                                                     Notes & comments

det rk0                                                     Detach the OS8 boot disk from rk0
att rk1 ../imagefiles/rsh/multos8.rk05                      Attach the MULTOS8 boot disk to rk1
att rk0 ../imagefiles/rsh/rsh.rk05                          Attach rk0 to a new file ‘rsh.rk05’. This will be the timeshare boot drive.
boot rk1                                                    Boot MULTOS/8 in maintenance mode

*Now booted to MULTOS/8 maintanance mode                     Looks and acts just like OS8 standard boot)

DIR RKA1:                                                   Directory listing of MULTOS/8 disk, first partition
DIR RKB1:                                                   Directory listing of MULTOS/8 disk, second partition
DIR RKA0:                                                   Will show nothing (or error) as the disk has not been ‘created’ yet.
ZERO RKA0:/Y                                           This initializes the RKA0: disk (rsh.rk05). You will be asked to confirm. **
COPY RKA0:<RKB0:*.*                           Copy all the files on the MULTOS/8 primary disk to the new disk.
DIR RKA0:                                                   Directory listing of RKA0:, now the same as RKA1:

DATE 23-JUN-70                                        Only dates in the ’70s work. This is needed to boot MULTOS8.
R MULTOS                                                  It will ask for a time in hhmm format, then boot to timeshare mode.

<ctrl-h>                                                        Log in user 1. There is no password so just press enter to get the ‘.’ prompt.
DIR                                                                This will show RKA0: listing, which is now the boot/default drive for timeshare.
R SYSTAT                                                      A command to show system status.
R LOGOFF                                                    A command to log off the user.

With this I was able to play on terminal 1 (the console) as user 1. I don’t have serial terminals set up for my PiDP8,  so can’t check other terminals. That’s a future project.

Critical NOTE: The /Y option on the zero command is critical as it makes the new disk bootable. If you omit the /y flag, then MULTOS will simply stall after you try and log in.

Here’s a sample session:

.R MULTOS

HELLO !

THIS IS THE MULTOS/8 MULTI-USER OS/8 TIMESHARING SYSTEM
CREATED BY COMPUTER METHODS
7822 OAKLEDGE ROAD
SALT LAKE CITY, UTAH 84121, USA
PHONE 801-942-8000

PLEASE INPUT TIME IN 24-HOUR FORMAT
(E.G. 0925 FOR 9:25 AM AND 1935 FOR 7:35 PM):
1022

THANK YOU !

THE SYSTEM IS NOW  TIMESHARING

TYPE CONTROL/H TO LOG ON.

PLEASE LOGON

.R SYSTAT
MULTOS-8   SYSTEM STATUS       1-JUN-1978    AT    12:27:10

UNASSIGNED MEMORY    16 K     SWAPPING FIELDS  6    MAX NO. USERS   2

JOB       ACTIVE        BUSY

SELF        1             0       TTY  I/O WAIT AT PC 6651

LPT         1             1

MEMORY USEAGE

FIELD   JOB  REL FIELD

LPT QUEUE EMPTY

.