Saturday, June 15, 2019

Jenkins X: Monitor Tekton Pipelines deployments with Kibana


If something goes wrong, the first thing you do is look at the logs. For serverless Jenkins X it is a little problematic if some time has passed -  build pods have already died safely and jx get build logs only gives you:

error: no Tekton pipelines have been triggered which match the current filter

But what if you want to find the cause of some error or compare the current build log with yesterday?
There is a dashboard for Tekton which looks nice but persistent logs unfortunately are not supported yet (I hope this feature will appear in the future).



Happily there is an Elastic Stack. I installed the Kibana following the instructions from tekton-pipelines-elastic-tutorials.
Although there are an warning that it was tested on a Mac only, everything started and works fine on Ubuntu, for example.



Kibana allows not only to monitor Tekton Tasks or Pipelines (and not only) in real time but also to check one or another log for yesterday or any other day.



Wednesday, May 22, 2019

How To Install Jenkins X On Ubuntu VM

I hope these steps will be useful for those who want to try Jenkins X on a local VM without external IP (NAT). In my case this is VMware ESXi VM, Ubuntu 18.04.2 LTS, 10GB RAM.
For the Kubernetes installation, just follow the good instructions from here. 
My installation was different in that the to fix coredns Pending state, I’ve installed weave-net pod network instead of CALICO:


$ kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\ n')"


Few preparations were necessary before jx install: I killed much time getting ADDRESS column not empty in kubectl get ing output, using ClusterIP, NodePort and LoadBalancer in rotation, so I chose separate ingress install:

$ helm upgrade --install ingress --namespace=ingress --set \

  rbac.create=true,controller.kind=DaemonSet,controller.service.type=ClusterIP,controller.hostNetwork=true,controller.extraArgs.report-node-internal-ip-address=true stable/nginx-ingress

If you see:

Error: could not find tiller 

use these commands:

$ kubectl —-namespace kube-system create serviceaccount tiller

$ kubectl create clusterrolebinding tiller-cluster-rule —-clusterrole=cluster-admin —-serviceaccount=kube-system:tiller

$ kubectl —-namespace kube-system patch deploy tiller-deploy -p ‘{“spec”:{“template”:{“spec”:{“serviceAccount”:”tiller”}}}}’ 

$ kubectl taint nodes —all node-role.kubernetes.io/master- 

   Then I used nfs provider to get persistent volumes (nfs server is required):

$ helm install stable/nfs-client-provisioner --set nfs.server=NFS_SERVER_IP --set nfs.path=NFS_SHARE_PATH
       
Now it's ready for install:

$ jx install --provider=kubernetes --external-ip=YOUR_VM_IP --ingress-service=ingress-nginx-ingress-controller --ingress-deployment=ingress-nginx-ingress-controller --ingress-namespace=ingress --on-premise

Answer "no" when jx will ask "No existing ingress controller found in the kube-system namespace, shall we install one?"

After install, check that ADDRESS column is not empty in kubectl get ing output:

$ kubectl get ing
NAME                HOSTS                                    ADDRESS      PORTS   AGE
chartmuseum         chartmuseum.jx.10.2.20.46.nip.io         10.2.20.46   80      42h
deck                deck.jx.10.2.20.46.nip.io                10.2.20.46   80      42h
docker-registry     docker-registry.jx.10.2.20.46.nip.io     10.2.20.46   80      42h
hook                hook.jx.10.2.20.46.nip.io                10.2.20.46   80      42h
jx-jx-app-jenkins   jx-jx-app-jenkins.jx.10.2.20.46.nip.io   10.2.20.46   80      41h
monocular           monocular.jx.10.2.20.46.nip.io           10.2.20.46   80      42h
nexus               nexus.jx.10.2.20.46.nip.io               10.2.20.46   80      42h
tide                tide.jx.10.2.20.46.nip.io                10.2.20.46   80      42h

  Check the local hook ingress connectivity:

$ curl -S hook.jx.10.2.20.46.nip.io

Now you can import your local project to Jenkins X with jx import or add some predefined projects with jx create.  
Since VM does not have an external IP,  webhooks that Jenkins prescribed on a Github will not work. There are several solutions here - for example ultrahook.  Replace Jenkins X webhooks URL for Staging, Production and your Dev repositories on Github with ultrahook URL and run with an ingress hook URL:

$ ultrahook github http://hook.jx.10.2.20.46.nip.io/hook

To get Jenkins X console working (it only makes sense if you use Jenkinsfile in some project),  needs to install Jenkins App first (choose all required plugins):

$ jx add app jx-app-jenkins


and run it:

$ jx console

  Jenkins Console: http://jx-jx-app-jenkins.jx.10.2.20.46.nip.io

After opening this URL in browser, you'll see usual Jenkins login page. Surprisingly, the admin password you wrote down after installation is not suitable here. Maybe I missed something, but I had to run the following commands to find out the password:

$ go get github.com/mfuentesg/ksd

and then look for the jenkins-admin-password field:

$ kubectl get secret jx-jx-app-jenkins -o yaml | ksd

And you should see the familiar interface after login:










Tuesday, November 6, 2018

Building ripgrep on illumos

Last night I tried to build ripgrep (grep on steroids). Of course I used Nexenta, but everything below will work for any illumos distribution. ripgrep is written in Rust and we can get the latest Rust version with pkgin (many thanks to Jonathan Perkin):


$ pkgin search rust
 ...
 rust-1.30.0 =        Safe, concurrent, practical language
 ...
$ pkgin install rust
$ export PATH=$PATH:/opt/local/bin

Building ripgrep:


$ git clone https://github.com/BurntSushi/ripgrep
$ cd ripgrep
$ cargo build --release
  Updating crates.io index

   warning: spurious network error (2 tries remaining): no Content-Type header in response; class=Net (12)
   warning: spurious network error (1 tries remaining): no Content-Type header in response; class=Net (12)
   error: failed to update registry `https://github.com/rust-lang/crates.io-index

 Caused by:
   failed to fetch `https://github.com/rust-lang/crates.io-index

 Caused by:
   no Content-Type header in response; class=Net (12)

Bang! If you have such error:  clone crates.io-index git to the .cargo directory in your $HOME:


$ cd ~/.cargo
$ git clone --bare https://github.com/rust-lang/crates.io-index.git

Then create .cargo/config file:


$ cat .cargo/config
[registry]
index = "file:///home/alhazred/.cargo/crates.io-index.git"

Now build will successful:

$ cargo build --release
...
$ ./target/release/rg --version
ripgrep 0.10.0 (rev fb62266620)
-SIMD -AVX (compiled)
+SIMD -AVX (runtime)

Now you can compare the ripgrep speed and performance with usual grep. See Andrew Gallant's Blog for more information. .

Wednesday, October 31, 2018

New v9os iso and repository archive


Well, two years later, I decided to upgrade v9os. As a result, after the noise of the servers under my ear I have a little bad hearing at the moment :). But iso was rebuilt with spring changes from the illumos (unfortunately the system is panicking after later commits, I hope to solve this out later).
Some things were fixed, some components were updated, pkg and caiman were moved to python 2.7. I’ve a more powerful server now and I hope that further updates will be forthcoming in this winter.
Site design was also a bit updated.

Wednesday, June 27, 2018

Unleashed - an operating system fork of illumos


If someone looking for the operating system with ZFS, DTrace and other illumos-buns, system that has been a bit cleared from the some outdated SunOS-code, with modern utilities and libraries, and most importantly, with simple contribution process (hey, illumos-gate refugees) - here it is: Unleashed-OS.


Notes for the first release.

Sunday, December 18, 2016

How to reset Solaris 11 root password in a minute using kmdb

Following a bit outdated, ufs-based method from "Using kmdb to reset lost Solaris root password" article, we still use uiomove break point. Apply kmdb and single-mode options to the grub kernel line and boot the machine. Wait for the SINGLE USER MODE message,  type "root" for the user name. Now wait a minute until smf services run and then switch to kmdb prompt (F1-a) right after "Enter root password (control-d to bypass):" message.



































Set break point at uiomove+0xc and type :c to continue. You will need to step forward by pressing :u and :c two or three times and look for the state where zfs`dmu_read_uio_dnode calls zfs`dmu_read_uio_dbuf and finally uiomove:



































On the screenshot below we found "/etc/shadow" data in ffff810042207c00. Now all what we need is to change the first character of the root password to ":" and, after removing break point and continue, we get the root shell prompt:

































Now you can reset the password and reboot.

Saturday, July 30, 2016

Meet v9os, a minimalist illumos-based SPARC distro

This summer I decided то build something compact and modern for my two SunFire servers using IPS. Meet v9os - minimalist illumos-based SPARC distro.
On this stage, actually it's an OmniOS SPARC version, because the iso was mostly created using omnios-build, so I would like to express my deepest gratitude to OmniOS developers.

The iso was tested on SunFire v240. I would be grateful for any feedback on what works, and how it boots on other SPARC servers. I'm looking for the hosting for the IPS repository, for now you need to download packed IPS repo from the sourceforge and put it somewhere on installed system. I forgot to fix timezone installer issue in this iso, so please just type “Continue” on timezone screen while installing. Root password is solaris. This is a beta release and it lacks yet some packages like java, gcc 5. 

Special thanks to Martin (I used OpenSXCE on the first step) and Igor for the SPARC support in illumos.

Friday, April 29, 2016

Firefly failsafe image was updated

The following changes have been made:

  1. smf removed completely
  2. freebsd bootloader from the illumos loader project
  3. busybox provides init and many others commands
  4. network configuration script (/etc/init.d/netconfig) is included
  5. it's a possible to copy files over network using scp
  6. console has white background color
  7.  image size now is 58 MB


Wednesday, December 9, 2015

The Linux “free” command for Solaris/illumos


I never understood why Solaris does not contain a simple way to checking the amount of used/free memory and swap.  On Linux you can check the used and available space of physical memory and swap by using "free" command.  Here's the simple analogue for solaris/illumos, based on Brendan Gregg's swapinfo. Unlike Linux version, “cached" means here memory used by the ZFS ARC Cache. Many thanks to Sebastian Wiedenroth, who has made significant changes and continues development.

# uname -s
SunOS
# free
           total       used       free     locked     kernel     cached
Mem:        128G       916M      87.0G          0      17.0G      23.1G
Swap:      75.9G      1.26G      74.7G

Saturday, September 19, 2015

GPT fdisk on illumos

Is there an alternative to the illumos native fdisk? Yes, for example, GPT fdisk from Rod Smith. It includes the gdisk, cgdisk, sgdisk, and fixparts programs. Site mentions only usage on Linux, FreeBSD, Mac OS X and Windows, but it is possible that GPT fdisk may be suitable on illumos-based distributions. It requires popt library (OmniOS contains it) and latest version built with no problems, except for a small fix for ncurses.
root@test:# ./gdisk /dev/dsk/c2t3d0p0
GPT fdisk (gdisk) version 1.0.0

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/dsk/c2t3d0p0: 10485760 sectors, 5.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): F24F914A-AD82-D84D-B0FC-81351563B356
Partition table holds up to 9 entries
First usable sector is 34, last usable sector is 10485726
Partitions will be aligned on 1-sector boundaries
Total free space is 222 sectors (111.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1             256        10469342   5.0 GiB     BF01  zfs
   9        10469343        10485726   8.0 MiB     BF07

Command (? for help): i
Partition number (1-9): 1
Partition GUID code: 6A898CC3-1DD2-11B2-99A6-080020736631 (Solaris /usr & Mac ZFS)
Partition unique GUID: F9BEC73D-23D3-C168-E74A-D60ED3EC565D
First sector: 256 (at 128.0 KiB)
Last sector: 10469342 (at 5.0 GiB)
Partition size: 10469087 sectors (5.0 GiB)
Attribute flags: 0000000000000000
Partition name: 'zfs'

Command (? for help):
b       back up GPT data to a file
c       change a partition's name
d       delete a partition
i       show detailed information on a partition
l       list known partition types
n       add a new partition
o       create a new empty GUID partition table (GPT)
p       print the partition table
q       quit without saving changes
r       recovery and transformation options (experts only)
s       sort partitions
t       change a partition's type code
v       verify disk
w       write table to disk and exit
x       extra functionality (experts only)
?       print this menu

Command (? for help):

Monday, July 27, 2015

Building own failsafe image using firefly scripts

I've put some distro-const scripts to the github repo at https://github.com/alhazred/firefly for anyone who wants to build their own illumos-based failsafe image. Current stuff was tested on OmniOS but also should work on OpenIndiana. Anyway the code can be simplified, so any pull-requests and customization are welcome.

Jenkins X: Monitor Tekton Pipelines deployments with Kibana

If something goes wrong, the first thing you do is look at the logs. For   serverless Jenkins X it is a little problematic if some ti...