Tuesday, November 6, 2018

Building ripgrep on illumos

Last night I tried to build ripgrep (grep on steroids). Of course I used Nexenta, but everything below will work for any illumos distribution. ripgrep is written in Rust and we can get the latest Rust version with pkgin (many thanks to Jonathan Perkin):

$ pkgin search rust
 rust-1.30.0 =        Safe, concurrent, practical language
$ pkgin install rust
$ export PATH=$PATH:/opt/local/bin

Building ripgrep:

$ git clone https://github.com/BurntSushi/ripgrep
$ cd ripgrep
$ cargo build --release
  Updating crates.io index

   warning: spurious network error (2 tries remaining): no Content-Type header in response; class=Net (12)
   warning: spurious network error (1 tries remaining): no Content-Type header in response; class=Net (12)
   error: failed to update registry `https://github.com/rust-lang/crates.io-index

 Caused by:
   failed to fetch `https://github.com/rust-lang/crates.io-index

 Caused by:
   no Content-Type header in response; class=Net (12)

Bang! If you have such error:  clone crates.io-index git to the .cargo directory in your $HOME:

$ cd ~/.cargo
$ git clone --bare https://github.com/rust-lang/crates.io-index.git

Then create .cargo/config file:

$ cat .cargo/config
index = "file:///home/alhazred/.cargo/crates.io-index.git"

Now build will successful:

$ cargo build --release
$ ./target/release/rg --version
ripgrep 0.10.0 (rev fb62266620)
-SIMD -AVX (compiled)
+SIMD -AVX (runtime)

Now you can compare the ripgrep speed and performance with usual grep. See Andrew Gallant's Blog for more information. .

Wednesday, October 31, 2018

New v9os iso and repository archive

Well, two years later, I decided to upgrade v9os. As a result, after the noise of the servers under my ear I have a little bad hearing at the moment :). But iso was rebuilt with spring changes from the illumos (unfortunately the system is panicking after later commits, I hope to solve this out later).
Some things were fixed, some components were updated, pkg and caiman were moved to python 2.7. I’ve a more powerful server now and I hope that further updates will be forthcoming in this winter.
Site design was also a bit updated.

Wednesday, June 27, 2018

Unleashed - an operating system fork of illumos

If someone looking for the operating system with ZFS, DTrace and other illumos-buns, system that has been a bit cleared from the some outdated SunOS-code, with modern utilities and libraries, and most importantly, with simple contribution process (hey, illumos-gate refugees) - here it is: Unleashed-OS.

Notes for the first release.

Sunday, December 18, 2016

How to reset Solaris 11 root password in a minute using kmdb

Following a bit outdated, ufs-based method from "Using kmdb to reset lost Solaris root password" article, we still use uiomove break point. Apply kmdb and single-mode options to the grub kernel line and boot the machine. Wait for the SINGLE USER MODE message,  type "root" for the user name. Now wait a minute until smf services run and then switch to kmdb prompt (F1-a) right after "Enter root password (control-d to bypass):" message.

Set break point at uiomove+0xc and type :c to continue. You will need to step forward by pressing :u and :c two or three times and look for the state where zfs`dmu_read_uio_dnode calls zfs`dmu_read_uio_dbuf and finally uiomove:

On the screenshot below we found "/etc/shadow" data in ffff810042207c00. Now all what we need is to change the first character of the root password to ":" and, after removing break point and continue, we get the root shell prompt:

Now you can reset the password and reboot.

Saturday, July 30, 2016

Meet v9os, a minimalist illumos-based SPARC distro

This summer I decided то build something compact and modern for my two SunFire servers using IPS. Meet v9os - minimalist illumos-based SPARC distro.
On this stage, actually it's an OmniOS SPARC version, because the iso was mostly created using omnios-build, so I would like to express my deepest gratitude to OmniOS developers.

The iso was tested on SunFire v240. I would be grateful for any feedback on what works, and how it boots on other SPARC servers. I'm looking for the hosting for the IPS repository, for now you need to download packed IPS repo from the sourceforge and put it somewhere on installed system. I forgot to fix timezone installer issue in this iso, so please just type “Continue” on timezone screen while installing. Root password is solaris. This is a beta release and it lacks yet some packages like java, gcc 5. 

Special thanks to Martin (I used OpenSXCE on the first step) and Igor for the SPARC support in illumos.

Friday, April 29, 2016

Firefly failsafe image was updated

The following changes have been made:

  1. smf removed completely
  2. freebsd bootloader from the illumos loader project
  3. busybox provides init and many others commands
  4. network configuration script (/etc/init.d/netconfig) is included
  5. it's a possible to copy files over network using scp
  6. console has white background color
  7.  image size now is 58 MB

Wednesday, December 9, 2015

The Linux “free” command for Solaris/illumos

I never understood why Solaris does not contain a simple way to checking the amount of used/free memory and swap.  On Linux you can check the used and available space of physical memory and swap by using "free" command.  Here's the simple analogue for solaris/illumos, based on Brendan Gregg's swapinfo. Unlike Linux version, “cached" means here memory used by the ZFS ARC Cache. Many thanks to Sebastian Wiedenroth, who has made significant changes and continues development.

# uname -s
# free
           total       used       free     locked     kernel     cached
Mem:        128G       916M      87.0G          0      17.0G      23.1G
Swap:      75.9G      1.26G      74.7G

Saturday, September 19, 2015

GPT fdisk on illumos

Is there an alternative to the illumos native fdisk? Yes, for example, GPT fdisk from Rod Smith. It includes the gdisk, cgdisk, sgdisk, and fixparts programs. Site mentions only usage on Linux, FreeBSD, Mac OS X and Windows, but it is possible that GPT fdisk may be suitable on illumos-based distributions. It requires popt library (OmniOS contains it) and latest version built with no problems, except for a small fix for ncurses.
root@test:# ./gdisk /dev/dsk/c2t3d0p0
GPT fdisk (gdisk) version 1.0.0

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/dsk/c2t3d0p0: 10485760 sectors, 5.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): F24F914A-AD82-D84D-B0FC-81351563B356
Partition table holds up to 9 entries
First usable sector is 34, last usable sector is 10485726
Partitions will be aligned on 1-sector boundaries
Total free space is 222 sectors (111.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1             256        10469342   5.0 GiB     BF01  zfs
   9        10469343        10485726   8.0 MiB     BF07

Command (? for help): i
Partition number (1-9): 1
Partition GUID code: 6A898CC3-1DD2-11B2-99A6-080020736631 (Solaris /usr & Mac ZFS)
Partition unique GUID: F9BEC73D-23D3-C168-E74A-D60ED3EC565D
First sector: 256 (at 128.0 KiB)
Last sector: 10469342 (at 5.0 GiB)
Partition size: 10469087 sectors (5.0 GiB)
Attribute flags: 0000000000000000
Partition name: 'zfs'

Command (? for help):
b       back up GPT data to a file
c       change a partition's name
d       delete a partition
i       show detailed information on a partition
l       list known partition types
n       add a new partition
o       create a new empty GUID partition table (GPT)
p       print the partition table
q       quit without saving changes
r       recovery and transformation options (experts only)
s       sort partitions
t       change a partition's type code
v       verify disk
w       write table to disk and exit
x       extra functionality (experts only)
?       print this menu

Command (? for help):

Monday, July 27, 2015

Building own failsafe image using firefly scripts

I've put some distro-const scripts to the github repo at https://github.com/alhazred/firefly for anyone who wants to build their own illumos-based failsafe image. Current stuff was tested on OmniOS but also should work on OpenIndiana. Anyway the code can be simplified, so any pull-requests and customization are welcome.

Thursday, January 15, 2015

Get pid, user and command for the particular port in illumos

While the Solaris has the updated netstat with -u flag, illumos requires certain improvements in sockinfo, and possibly, rpcbind (tcpConnCreationProcess shows the old PID for rpcbind, until it did not become a demon). With help of pfiles is possible to obtain the desired information about pid for particular port, so I wrote a small pfiles-based utility which allows to determine the PID, user and command for the particular port:

# ./ptcpstat
PROTO           IP               PORT   PID    USER     COMMAND
--------  -------------------    ----- ------ -------- ------------------
AF_INET              25    692    root      /usr/lib/smtp/sendmail/sendmail -bl -q15m
AF_INET6  ::1                    25    692    root      /usr/lib/smtp/sendmail/sendmail -bl -q15m
AF_INET              587   692    root      /usr/lib/smtp/sendmail/sendmail -bl -q15m
AF_INET6  ::                     22    605    root      /usr/lib/ssh/sshd
AF_INET6  ::                     0     495    root      /usr/lib/inet/in.ndpd
AF_INET                520   637    root      /usr/sbin/in.routed
AF_INET                0     637    root      /usr/sbin/in.routed
AF_INET                10004 685    root      /usr/bin/python2.6 /usr/lib/pkg.depotd --cfg svc:/application/pkg/server:test
AF_INET                10000 9354   root      /usr/bin/python2.6 /usr/lib/pkg.depotd --cfg svc:/application/pkg/server:zfs
AF_INET6  ::ffff:     22    2315   root      /usr/lib/ssh/sshd
AF_INET6  ::ffff:       56027 2315   root      /usr/lib/ssh/sshd
AF_INET                80    1402   root      /usr/bin/python2.6 /usr/lib/pkg.depotd --cfg svc:/application/pkg/server:defaul
AF_INET6  ::ffff:     22    2314   root      /usr/lib/ssh/sshd
AF_INET6  ::ffff:       56027 2314   root      /usr/lib/ssh/sshd

Monday, May 19, 2014

How to prevent an IPS package from being upgraded

Recently I had a situation where some incorporating package depends on a package, say foo, version 3.1 and I had to wait for the update exactly to 3.2 and do not allow 3.1.1, 3.1.2, ...
"Freeze" option doesn't help here (may be should?) but it's a possible to use some versioning tricks in incorporating  manifest.
First option is to ensure that required version of 'foo' is published with trailing zero:


Then we can use "depend fmri=foo@3.1.0 type=incorporate" line in incorporating manifest and only upgrade to this version will be allowed.

More reliable option (and you don't need trailing zeros) is to use the timestamp in incorporate dependency:

    depend type=incorporate fmri=foo@3.1:20140507T021327Z

Only upgrade to version 3.1 with specific timestamp will be allowed here.

If foo@3.2 will be published and we want to wait until 3.3, then we can publish a new version of the incorporating package using next dependencies:

depend type=require fmri=foo@3.2
depend type=incorporate fmri=foo@3.2.0

P.S. Many thanks to @binarycrusader

Building ripgrep on illumos

Last night I tried to build ripgrep (grep on steroids). Of course I used Nexenta, but everything below will work for any illumos distribu...