OpenIndiana hipster 2014.1

July 10th, 2014

I haven’t been paying close attention the last few weeks and was surprised to discover that hipster stopped updating. So, I finally had time to go read the mailing list and discovered that they’d moved to a hipster-2014.1 repository that contains just the most recent packages.

Needed to set up my boxes at work to mirror this repository, so I created a new local mirror:

# mkdir /data/pkg/hipster-2014.1
# pkgrepo create /data/pkg/hipster-2014.1
# pkgrecv -s http://pkg.openindiana.org/hipster-2014.1/ -d /data/pkg/hipster-2014.1 '*'
# svccfg -s pkg/server add hipster-2014-1
# svccfg -s pkg/server:hipster-2014-1 addpg pkg application
# svccfg -s pkg/server:hipster-2014-1 setprop pkg/port=10087
# svccfg -s pkg/server:hipster-2014-1 setprop pkg/inst_root=/data/pkg/hipster-2014.1
# svccfg -s pkg/server:hipster-2014-1 addpg general framework
# svccfg -s pkg/server:hipster-2014-1 addpropvalue general/complete astring: hipster-2014.1
# svccfg -s pkg/server:hipster-2014-1 addpropvalue general/enabled boolean: true
# svccfg -s pkg/server:hipster-2014-1 setprop pkg/readonly=true
# svccfg -s pkg/server:hipster-2014-1 setprop pkg/threads=100
# svcadm refresh application/pkg/server:hipster-2014-1
# svcadm enable application/pkg/server:hipster-2014-1

Then, I have to fix apache on the box by editing proxy.conf:

# vi /etc/apache2/2.2/conf.d/proxy.conf

I add the following line:

ProxyPass /hipster-2014.1 http://pkg.example.com:10087 nocanon

Making the file look like:

ProxyRequests Off
ProxyVia Block
ProxyStatus On
ProxyPreserveHost Off
ProxyPass /dev http://pkg.example.com:10081 nocanon
ProxyPass /sfe http://pkg.example.com:10082 nocanon
ProxyPass /sfe-encumbered http://pkg.example.com:10083 nocanon
ProxyPass /local http://pkg.example.com:10084 nocanon
ProxyPass /legacy http://pkg.example.com:10085 nocanon
ProxyPass /hipster http://pkg.example.com:10086 nocanon
ProxyPass /hipster-2014.1 http://pkg.example.com:10087 nocanon
AllowEncodedSlashes NoDecode

Restart apache, set the publisher, and update:

# svcadm restart svc:/network/http:apache22
# pkg set-publisher -p http://pkg.example.com/hipster-2014.1 openindiana.org
pkg set-publisher:
  Updated publisher(s): openindiana.org
# pkg set-publisher -p http://pkg.example.com/hipster-2014.1 openindiana.org
pkg set-publisher:
  Updated publisher(s): openindiana.org
# pkg image-update
           Packages to install:   3
            Packages to update: 487
           Mediators to change:   2
       Create boot environment: Yes
Create backup boot environment:  No

DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                            490/490     7415/7415  674.6/674.6 13.5M/s

PHASE                                          ITEMS
Removing old actions                       2682/2682
Installing new actions                     3920/3920
Updating modified actions                10263/10263
Updating package state database                 Done
Updating package cache                       487/487
Updating image state                            Done
Creating fast lookup database                   Done
Reading search index                            Done
Building new search index                  1236/1236

A clone of openindiana-41 exists and has been updated and activated.
On the next boot the Boot Environment openindiana-42 will be
mounted on '/'.  Reboot when ready to switch to this updated BE.


---------------------------------------------------------------------------
NOTE: Please review release notes posted at:

http://wiki.openindiana.org/display/oi/oi_hipster
---------------------------------------------------------------------------

And reboot, and then you have a new hipster box.

Enlightenment (e16) 1.0.13 for Debian

June 22nd, 2014

I’m a big fan of Enlightenment DR16. Not that there’s anything really wrong with E17, I just prefer the older one to every other window manager I’ve used. The list is long.

However, e16 is getting removed from just about every distro out there. It’s still in Gentoo, but today I built a VM of Wheezy and sadly it’s gone. I’m pretty sure it disappeared from Fedora awhile ago. I’ve stopped using Fedora almost entirely so I can’t be sure. I just have some older releases in VMs now.

I had a friend once who was basically maintaining OpenLook olvwm and the XView libraries because that’s what he preferred. I am assuming he gave up by now given all the 64-bit issues that apparently exist with XView. I am starting to understand what it is like to go it alone.

I maintain a local portage overlay ebuild of 1.0.12+ for my Gentoo boxes (it has a patch that fixes some focus issues that seem to exist only in Java.)

Building GNU Screen 4.2.1 on OpenIndiana

April 30th, 2014

Surprisingly, a new version of GNU screen (4.2.1) was released recently, so I thought I’d build it on my OpenIndiana (hipster) box to see if worked, and ran into a bit of a problem:

gcc -c -I. -I.  -DETCSCREENRC='"/usr/local/etc/screenrc"' -DSCREENENCODINGS='"/usr/local/share/screen/utf8encodings"' -DHAVE_CONFIG_H -DGIT_REV=\""`git describe --always 2>/dev/null`"\" \
     -g -O2 socket.c
socket.c: In function 'ReceiveMsg':
socket.c:990:16: warning: assignment from incompatible pointer type [enabled by default]
socket.c:994:6: error: 'struct msghdr' has no member named 'msg_controllen'
socket.c:995:6: error: 'struct msghdr' has no member named 'msg_control'
socket.c:1007:14: error: 'struct msghdr' has no member named 'msg_controllen'
socket.c:1010:14: warning: assignment makes pointer from integer without a cast [enabled by default]
socket.c:1010:48: warning: assignment makes pointer from integer without a cast [enabled by default]
socket.c: In function 'SendAttachMsg':
socket.c:1801:6: error: 'struct msghdr' has no member named 'msg_control'
socket.c:1802:6: error: 'struct msghdr' has no member named 'msg_controllen'
socket.c:1803:8: warning: assignment makes pointer from integer without a cast [enabled by default]
socket.c:1807:3: warning: passing argument 2 of 'bcopy' makes pointer from integer without a cast [enabled by default]
In file included from os.h:83:0,
                 from screen.h:30,
                 from socket.c:42:
/usr/include/strings.h:46:13: note: expected 'void *' but argument is of type 'int'
socket.c:1808:6: error: 'struct msghdr' has no member named 'msg_controllen'
gmake: *** [socket.o] Error 1

Well, that didn’t work as hoped, but after looking at the source and doing a quick search, I realized I could get it to build (and work, based on my limited testing), using:

CFLAGS="-D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED=1 -D__EXTENSIONS__" ./configure --prefix=/usr/local

And then building with gmake, it works just fine. Looks like the project is alive again.

HBO Go on Linux

September 30th, 2013

HBO Go stopped working for me on Linux recently (actually it might not have been super recent, I don’t use it that often). Apparently a DRM problem with Flash 11.2.x on Gentoo (and other Linux versions I’m assuming), I was able to get it working again by emerging “media-libs/hal-flash” (version 0.2.0_rc1 I assume). I have no idea why HAL is “deprecated”.

The move away from Flash is going to eventually be OK, since it came with its own issues, particularly on 64 bit Linux. In the short term it has caused me a lot of problems

Setting up repository mirrors for local use

August 29th, 2013

I have a few OpenIndiana servers now, at home and at work. I’ve been working hard on clustering lately. To help with my work, in both places I’m busy setting up a local repository, something I can update periodically and then use it to update all my servers and containers. I’ve had a lot of success with a local http-replicator cache for Gentoo, and I hope this will prove useful as well. I’m not all that familiar with IPS anyway, so anything I can learn here would be useful. I sometimes miss the simplicity of SVR4 packages but they never added the features the other distributions were busy pioneering.

Based on some documentation I’ve been reading, to create the repository on my local machine, in a data zpool that I’d already created:

mkdir /data/pkg/dev
pkgrepo create /data/pkg/dev
pkgrecv -s http://pkg.openindiana.org/dev/ -d /data/pkg/dev '*'
pkgrepo rebuild -s /data/pkg/dev

To set up the dev pkg.depotd server:

svccfg -s pkg/server add dev
svccfg -s pkg/server:dev addpg pkg application
svccfg -s pkg/server:dev setprop pkg/port=10081
svccfg -s pkg/server:dev setprop pkg/inst_root=/data/pkg/dev
svccfg -s pkg/server:dev addpg general framework
svccfg -s pkg/server:dev addpropvalue general/complete astring: dev
svccfg -s pkg/server:dev addpropvalue general/enabled boolean: true
svccfg -s pkg/server:dev setprop pkg/readonly=true
svccfg -s pkg/server:dev setprop pkg/threads=100

Some of these commands are borrowed from Solaris 11 instructions. I still need to look up some of these properties and find out if they are documented. Before today I had only seen the pkg/port, pkg/inst_root and pkg/readonly properties.

To enable it:

svcadm refresh application/pkg/server:dev
svcadm enable application/pkg/server:dev

After all the depot servers are up and running, I added this to the /etc/apache2/2.2/conf/proxy.conf file:

ProxyRequests Off
ProxyVia Block
ProxyStatus On
ProxyPreserveHost Off
ProxyPass /dev http://pkg.example.com:10081 nocanon
ProxyPass /sfe http://pkg.example.com:10082 nocanon
ProxyPass /sfe-encumbered http://pkg.example.com:10083 nocanon
ProxyPass /local http://pkg.example.com:10084 nocanon
ProxyPass /legacy http://pkg.example.com:10085 nocanon
AllowEncodedSlashes NoDecode

After that is done, you need to set up the clients to use it. To replace the openindiana.org publisher:

pkg set-publisher -G '*' -g http://pkg.example.com/dev/ openindiana.org
pkg refresh --full

After that, I was able to do ‘pkg update’ as expected. I was able to mirror dev, sfe, sfe-encumbered and legacy without much trouble.

Now that I have that, I’ve created a local repository for development. This will be fun. I am wondering how to feed back some packages to the community. I should probably ask.

Tomcat 6.x latency on POST

July 22nd, 2013

We were having some problems with HTTP POST performance from the West coast to a tomcat server farm on the East coast. None of these problems showed up locally (between two hosts in the same datacenter.) The latency between West and East coast sites is on the order of 70ms. After much investigation, we (two of my coworkers and I) managed to reduce a post of a 32MB file from ~87 seconds to ~4 seconds with a combination of two things: changing socket.rxBufSize and changing /etc/sysctl.conf:

net.ipv4.tcp_mem=196608 262144 1048576
net.core.rmem_max = 67108864
net.core.wmem_max = 67108864
net.ipv4.tcp_rmem = 4096 1048576 67108864
net.ipv4.tcp_wmem = 4096 1048576 67108864
net.core.netdev_max_backlog = 250000

In this case I think tcp_rmem and rmem_max are the important settings here. One of the odd things about this was the TCP window wasn’t scaling up…. the most I was seeing before this was 38K or so. Not at all what I expected.

By just changing the Tomcat 6 server.xml buffer size we got the time down from 87 to 39 seconds. A further change of the sysctl parameters got us down to just over 4 seconds. The changes we made to the sysctl parameters were limited to the second argument to net.ipv4.tcp_rmem and net.ipv4.tcp_wmem. Those other parameters were set for a previous unsuccessful attempt to improve the situation.

Also, these values were arrived at unscientifically. There are almost certainly some better choices out there. Based on some testing with stock CentOS 5.x scp, I think I should be able to get it to 2 seconds or better.

Pacemaker cluster on OpenIndiana

April 3rd, 2013

Today I finally got my NAS cluster working on the 3 OI 151a7 boxes I have set up in the lab at work for this purpose.

I took the work Mike Rowell had done on Linux-HA clustering and tweaked it a bit, and then added an OCF resource script to handle zpools and SCSI-3 persistent reservations groups. The idea of doing this without SCSI-3 PRG seemed like a disaster waiting to happen, but the sg3_utils that are available seem to work fine with our EMC Vplex.

I’ll have to document the details, perhaps in a future post here or something. It’s far from polished yet, but I can move zpools between nodes in the cluster with crm and crm_resource, and they’re exported as expected. Maybe it’ll help someone else trying to do HA SAN work with OI.

I’m hoping to make this work with OmniOS too.

sys-fs/eudev

January 19th, 2013

Looks like it’s finally time to migrate to sys-fs/eudev. udev 197 was unmasked today on amd64, and I would rather not go down that path.

At least I have an alternative, so I don’t have to go it alone.

I ended up having to use kmod, instead of module-init-tools. modprobe -l no longer works. I guess that feature is deprecated, and I’m supposed to use find in /lib/modules now. That seems less convenient. Other than that they seem the same.

ZFS assertion failed for SAN devices

January 8th, 2013

This was frustrating:

# zpool import nas2
Assertion failed: rn->rn_nozpool == B_FALSE, file ../common/libzfs_import.c, line 1086, function zpool_open_func

Discovered that I needed to tell zpool where to look for the devices:

zpool import -d /devices/scsi_vhci nas2

There was a time when I would have known that already, if we still used Solaris in any way at work. I wish the error message was more descriptive. Considering asking about it on the openindiana or illumos mailing lists.

Apache proxy relay OpenIndiana pkg install

May 22nd, 2012

Hi,

This took me a bit of time to figure out. I needed to update some OpenIndiana boxes from 151a to 151a4 and they had no direct access to the internet. I did, however, have a box that I could relay the requests through. I first figured this method out, and then later I found a hint that there is a better way to do it. I don’t have the details on the “new” way written down, but I will include this other Apache virtual host proxy thing in case someone finds it useful.

I ended up adding this to the 00_default_vhost.conf file (on Gentoo, but a similar incantation would work on Apache 2.2 elsewhere I’m sure):

Listen 8080
NameVirtualHost *:8080

<virtualhost *:8080>
        ServerName relay.example.com:8080
        Include /etc/apache2/vhosts.d/default_vhost.include

        <ifmodule mpm_peruser_module>
                ServerEnvironment apache apache
        </ifmodule>
       
        ProxyRequests On
        ProxyVia Block
        ProxyStatus On
        ProxyPreserveHost Off
        ProxyPass /dev/ http://pkg.openindiana.org/dev/
        ProxyPass /legacy/ http://pkg.openindiana.org/legacy/
        ProxyPass /sfe/ http://pkg.openindiana.org/sfe/
        ProxyPass /sfe-encumbered/ http://pkg.openindiana.org/sfe-encumbered/
        ProxyPassReverse /dev/ http://pkg.openindiana.org/dev/
        ProxyPassReverse /legacy/ http://pkg.openindiana.org/legacy/
        ProxyPassReverse /sfe/ http://pkg.openindiana.org/sfe/
        ProxyPassReverse /sfe-encumbered/ http://pkg.openindiana.org/sfe-encumbered/
       
        AllowEncodedSlashes NoDecode

</virtualhost>

This was cobbled together from various sources. I wish I’d kept some references to them. The tricky bit for me was the AllowEncodedSlashes, an Apache directive I had never heard of before. Before that it can’t find packages because it translates %2F to /, and therefore the OpenIndiana pkg.depotd server can’t find the right file.

After that you just point at this with pkg set-publisher:

pfexec pkg set-publisher -p http://relay.example.com:8080/dev/ openindiana.org

You will also need to clean up the old openindiana.org publisher if I recall correctly.

Anyway, it kept me from having to copying all of OpenIndiana and setting up a repository locally.