THE QUEST FOR THE PERFECT ARCH LINUX CONTAINER – #3 – makepkg

This is a small update from the previous post.

It’s now possible to build packages from within our humble container. I just did a commit with the following main additions:

  • install a couple of packages from the official repositories
  • create an ‘archer’ user (which is gonna be our general main non-root admin user)
  • disable password authentication for archer whenever using sudo
  • install cower, so we can easily and manually add packages from the AUR
  • install pacaur, so we can easily and automatically add packages from the AUR

As a small POC, I just updated the translate-shell package.

Everything is working okay, however this environment is still far from being called home. For now, it is just acting as a disposable container. There are still several inconveniences which I am hoping to fix and share in the next posts.

THE QUEST FOR THE PERFECT ARCH LINUX CONTAINER – #3 – makepkg

THE QUEST FOR THE PERFECT ARCH LINUX CONTAINER – #2 – Hello World

This post is part of an ongoing series about setting up a suitable Arch Linux environment for everyday use in a container. Previous post.

Recall: we’re starting from base/devel.

First things first…we gotta create a Dockerfile!

And now let’s make it more tidy and organized for deployment by using a docker-compose.yml file:

NOTE: The repo you see in the screencast is this one. The docs used as reference to create the docker-compose.yml file are here.

THE QUEST FOR THE PERFECT ARCH LINUX CONTAINER – #2 – Hello World

THE QUEST FOR THE PERFECT ARCH LINUX CONTAINER – #1

This post is part of an ongoing series about setting up a suitable Arch Linux environment for everyday use in a container. Previous post.

Container technology

No discussion: docker (wiki). For the sake of completeness, other options include: systemd-nspawn (although it is limited to Linux hosts running systemd), runC and low-level wizardry with LXC.

Where to start from?

We have to choose a image to start FROM. A few Google and Docker Hub searches yield the following options:

There are lots of options other than these, and there is also the option of rolling our own base. After a few hours of code inspection, I’ve decided to go with base/archlinux.

There’s also another variant: bash/devel. It just adds another layer atop base/archlinux, with the packages from the base-devel group, which is exactly what I would have done anyway, therefore I am starting with base/devel.

THE QUEST FOR THE PERFECT ARCH LINUX CONTAINER – #1

The quest for the perfect Arch Linux container – #0

Dear all,

This innocent blog is getting lots of attention from its humble author these days. This time, he is all in to start a new blog series about — heck, you already read it in the title.

It’s worth it to highlight that creating a series is very costly, both by being time-consuming and in terms of documenting its steps. Previous series included:

Motivation

To truly understand motivation, some context is needed. So, briefly: I’m a former Arch Linux community contributor, having spent almost 2 years using the OS and diving into its ecosystem[1], interacting with my peers everyday.  The next logical step would be to apply to become an official developer or a Trusted User (a person responsible to maintain packages and ensure quality in the [community] repo), and in fact I almost did that; however, at the time, I had to interrupt my involvement with the community because I participated in a study-abroad program. By the same time, I migrated to MacOS, which I’m using nowadays.

However — heck — migrating to MacOS doesn’t imply an automatic migration to the Apple ecosystem and culture. In fact, my (let’s say) baseline is still Arch.

That said, I am still using MacOS. What the hell? Am I that hypocritical and insincere?

So, let’s review the options (note to the reader: I tried them all):

  1. Replace MacOS with Arch Linux in my Macbook.
  2. Dual boot MacOS and Arch Linux in my Macbook.
  3. Use only MacOS in my Macbook.

I want start with the second one: it’s classical and very tempting, and it seems to solve all the problems. However, it could also be a mistake. “Less is more” and “do one thing, and do it well” are two Unix philosophies that won’t agree with this. I’ve tried this approach twice and they both failed. Why? Maintenance burden and lack of utilization of one of the systems. If you have to log in everyday in your workstation and then choose one operating system to boot in, you likely won’t boot into the other one for the rest of the day. And for the rest of the week. And for the rest of the month. Therefore, you end up with one operating system you likely won’t use. It would be better to invest this energy in just one OS, then.

The first option (using Arch only) also seems reasonable, but there are some moments in your life where Linux doesn’t suit it all. Ever needed to open a DRM’ed Acrobat Reader document? To use your printer or scanner with no-hassle? I could enumerate a few common things that might be needed to be done every once in a while. Some of them can be done in Linux with some effort (and time); others are simply not supported or even possible. That’s when having a MacOS (or a Windows) box helps. They are commercially supported OSes. It doesn’t matter whether you hate their lack of open-source: they are business standards.

The workaround for the previous paragraph is to have an Arch OS with a Windows VM. Yes, it definitely works. However, why would you do that if you have a Macbook? It’s way more efficient (especially hardware-wise) to use a native MacOS. That said, option #1 is totally okay, as long as you have a second computer with Windows to help you every now and then, or as long as you don’t need commercial support at all, ever.

And then we get to option #3: to use MacOS only. Here are some side effects I experienced with that approach:

  • I stopped interacting with the Arch community everyday, and I didn’t replace this interaction with anything else — to be honest, Stack Exchange is a good community to interact with, but it is not social — it is a strictly technical, Q & A forums.
  • I stopped learning everyday. OK, let’s be more precise: of course I still learn everyday; however, I used to learn everyday way more with Arch (or Linux, for that matter) than I do with MacOS. Since most things work out-of-the-box and with no stress in MacOS, I feel like a spoiled child with no problems to solve, no programs to compile, no configurations to tweak. This is harmful to a developer / programmer who wants to grow more and more — you can increase your expertise and ability only if you fail and then recover constantly.
  • I stopped putting the OS in the way, so I stopped worrying about a lot of things that I (as a programmer, developer) should worry about.
  • I stopped contributing to open source. Again, being more precise: I still contribute to open source, but way less than what I was used to before. An easy way to see that is with bug-wrangling and bug reporting: nowadays I mostly don’t care whether a given piece of software is or isn’t working. If it is not, [in general] I simply move to an alternative, like a spoiled kid who throws her smartphone away and gets a new one from her parents. Back into a few years ago, I used to care about most software I had installed on my computer, tracked bugs, tried to fix them or to find solutions for them. I even wrote blog posts about them and ranted about them in social media.
  • This might sound surprising, but it is also worth it to be highlighted: I decreased my serendipity massively. Heck, this blog is named after this awesome and beautiful word, yet I am saying I lost a big part of it thanks to my change to MacOS only. Well, of course the internet is full of articles and news, as life is full of people and friends. However, not all news and not all people are good. It’s like an expression we have in Portuguese: “diga com quem andas e te direi quem és”.

On the other hand, there is no need to dual boot Arch with MacOS, because MacOS is very good these days, even if you ignore all the fluff and distraction. Terminal.app and HomeBrew are the only utilities you need 99% of the time.

That’s how I came up with the container idea: it is an hybrid between “having MacOS only” and “still having Arch around when I might need it, but not with a full, overblown VM”. And it still provides a way of getting involved again with the community.

There are lots of issues coming through: this journey won’t be trivial. But I think it can be done and it will be a good experience, so it is worth it to spend some time to create a series about it.

That said, here is a disclaimer: sometimes I start a series but don’t continue it. But I am confident this time I will see you soon in the next post. Until the next time!

Footnotes

[1]: it’s not just the operating system. There are lots of idiosyncrasies and philosophies surrounding the Arch environment.

The quest for the perfect Arch Linux container – #0

Impressões de migração do Windows para o Elementary OS (não-superusuários)

O título não é lá essas coisas, mas a ideia é bastante simples: decidi tentar migrar o workflow de computação dos meus pais para o Linux. Eles tipicamente utilizam Windows mas, com a exceção de um ou dois programas, todos os outros estão também disponíveis para o Linux.

Na primeira vez que tentei fazer isso, em meados de 2012 ou 2013, foi um fracasso. Em primeiro lugar, eu estava aprendendo sobre Linux e, em particular, sobre a enorme diversidade de distros que estavam disponíveis na época. Acabei optando pelo Ubuntu (…12.04 ou 12.10). Só que foi um desastre. Eu tinha o Ubuntu instalado no meu próprio Desktop, com a interface do Gnome 2 (heck, eu sequer sabia o nome da interface). Quando instalei o Ubuntu no PC dos meus pais, do nada apareceu outra interface. Depois eu descobriria que a interface em questão era o Unity, que eu (e eles) detestaram, mas eu fiquei bastante perdido para configurá-lo com o Gnome 2, com o qual eu estava acostumado.

Alguns programas eram esquisitos (para eles) — por exemplo, evolution como leitor de emails em vez do thunderbird / outlook. O dock na lateral esquerda também não era intuitivo. Nem mesmo o botão de desligar…enfim, esse setup chegou a durar algumas semanas, mas não deu certo, e acabaram voltando para o Windows. Enquanto isso, eu começaria a minha quest (parte da qual está documentada nesse blog) de testar dúzias de distros diferentes e formar novas opiniões.

Alguns anos depois, agora em 2017, decidi fazer uma nova tentativa. Há cerca de dois meses, instalei o Elementary OS e deletei o Windows (10) do PC deles (yeah, nada de dual boot, não dá pra fazer transformação de mentirinha: ou vai de uma vez só, ou não vai).

Algumas razões para escolher o Elementary OS em vez de zilhões de outras distros incluem:

  • não é o Ubuntu (yaaaaay!)
  • é beginner-friendly e user-friendly
  • o foco da distro está em usabilidade e design, e não em configurações ou linha de comando ou…
  • utiliza um package manager bastante standard (o apt). Aqui existem vantagens e desvantagens mas, nesse contexto, isso é uma vantagem, principalmente pela quantidade de pacotes já disponíveis no ecossistema deb;
  • o seu desktop environment padrão é o pantheon, que é realmente excelente, e bastante simples e polido para usuários novatos.
  • o plank (que é o seu dock) também é bastante refinado — já foi bem ruinzinho, mas hoje em dia está ótimo.

Baseado nessa lista, outras distros promissoras nessa categoria incluem o Linux Mint, o Zorin OS e o PC-BSD. Por que não o Zorin OS? Porque ninguém usa essa joça. Se fosse pra ser indie, então eu instalaria o Gentoo ou o Arch e faria uma customização massiva para deixá-los user-friendly, então tchau tchau pro Zorin.

O PC-BSD não foi considerado porque ele não é Linux, é um mero spin-off do FreeBSD, e os caras renomearam o projeto recentemente para TrueOS, eu não estou interessado em projetos que estão mudando de nome e FreeBSD para Desktop não é lá grande coisa.

Dito isso, o Linux Mint poderia ser uma segunda opção. O motivo de ter optado pelo Elementary OS é porque eu perdi a confiança no projeto ao longo dos anos (Linux Mint já foi uma das minhas distros principais), por uma série de motivos que não valem a pena ser mencionados aqui; enquanto que a minha confiança no Elementary OS só veio aumentando (eu cheguei a conhecer um dos fundadores no SCALE14x, você percebe que os caras têm uma enorme paixão pelo que fazem, é fantástico).

Vale dizer que o Elementary OS não é lá grande coisa, ele possui uma porção de bugzinhos, que são fáceis de ser notados à medida que você usa o sistema, mas ele é razoavelmente estável, suficientemente usável. Por exemplo, o pantheon-files, que é seu file manager padrão, possui um bug ridículo na ação de renomear pastas, ela só funciona se você apertar ENTER (clicar com o mouse fora da área de escrita implica em retornar ao nome original). Eu optei por instalar o Nautilus, que inclusive possui integração com o Dropbox, o que é um bônus.

O Geary, que agora é chamado de Pantheon Mail, também é buguento. Se você adicionar uma conta de email dele mas depois se arrepender e quiser deletá-la, você…errr, simplesmente não pode fazer isso. (de forma user-friendly, ao menos). Bom, o Thunderbird está aí para essas horas sombrias.

Existem vários bugzinhos nesse estilo, coisas que são inconvenientes e que claramente não deveriam estar ali, mas que estão. Aparentemente a comunidade está ciente disso, ao menos em alguns deles, então é provável que eles sejam consertados na release 1.0 (a release atual é a 0.4).

Se você superar os bugzinhos e não surtar com isso, podemos ir para a segunda etapa. Alguns tweaks e atitudes que se provaram boas incluem:

  • unattended-upgrades, apt-cron: para manter o sistema atualizado automaticamente;
  • redshift para manter a saúde dos olhos (ninguém liga pra esse negócio de espectro vermelho e azul, mas as pessoas realmente deveriam se importar mais com isso, especialmente nesses tempos atuais em que o uso de dispositivos eletrônicos é massivo);
  • dropbox para backups (infelizmente nenhum cliente do google drive está disponível gratuitamente para Linux, até hoje), com integração com o Nautilus;
  • VMWare Player para manter um Windows “de emergência” para os programas que não estão disponíveis para Linux;
  • zramswap e diminuição do valor de swappiness do kernel para dar uma aliviada no disco rígido. O valor default é 60; eu diminuí para ~20–30. É uma faixa de valores razoável para um desktop com 4GB de RAM;
  • preload para carregar programas mais rapidamente na inicialização;
  • indicator de velocidade de rede no wingpanel;
  • VLC como reprodutor padrão de vídeos;
  • não instalar o vim. Ele insiste em ser o visualizador de arquivos padrão para um monte de formatos. Até mesmo o pacote vim-nox atrapalha.
  • desabilitar serviços desnecessários no systemd (systemctl list-unit-files; systemctl disable <unit>). Sim, a release atual do elementary OS usa o systemd.

Também convém instalar pacotes relacionados a NTFS e V-FAT, para pen-drives. Acho que um dos pacotes se chama dosfstools ou algo similar.

Tenho rodado esse setup há dois meses e ele tem se mostrado bem estável e razoável. Estou satisfeito. E, o melhor de tudo: a manutenção é mínima, não precisa de anti-vírus, e é rápido. Trava muito pouco, se comparado com o Windows 10. Todos os pacotes estão sendo gerenciados pelo apt: não há nenhum software instalado localmente, por fora. Isso diminui drasticamente qualquer necessidade de manutenção. Todos os scripts que estão rodando automaticamente (que não foram mencionados aqui) estão sendo gerenciados pelo systemd (timers!) e/ou pelo anacron.

Alguns projetos futuros incluem:

  • gerenciar e configurar o sistema com o saltstack;
  • minerar bitcoin;
  • tentar utilizar o wine em vez do VMWare para os programas que não estão disponíveis para Linux;
  • adicionar um firewall ecochato;
  • backups automáticos (btrfs snapshots? Periodic rsync?).
Impressões de migração do Windows para o Elementary OS (não-superusuários)

Journalog #3: eudyptula – challenge #1

This is not really a challenge; I’ve done it before. Anyways, now I have the insight about what’s happening behind-the-scenes, and also it’s very easy to recreate the environment and obtain the tools I need to do it.

The first challenge is very simple, as rephrased in my own words:

Write a hello world linux kernel module for your current kernel (supposedly 2.6+), which writes “Hello World!” to your kernel debug log level, upon its initialization.

Tools and programs you’ll need: vim, make, insmod, lsmod, rmmod. And, later on, mutt or sendmail or similar to send your work to little.

Overview

It’s better to start with the Makefile. Add at least two targets to it: all and clean. Refactoring variables (for example, paths) is also nice, if you see yourself repeating the same strings in both targets.

Later, you will create a hello.c file which should be very simple in nature. It should have two functions, one for initialization and one for cleanup for the time the module is unloaded. It will probably also have a couple of standard macros for modules.

Testing your work should also be very straightforward: (i) make, (ii) insmod hello.ko, (iii) rmmod hello. And see your log messages with dmesg | tail.

After everything is done, you can use mutt to send your sources to little, either interactively (hint: use ‘a’ to add attachments) or with a single command:

mutt -s "<id> challenge 1" -a hello.c -a Makefile little@eudyptula-challenge.org < <message>

where <message> is a file corresponding to the body or your email.

 

Journalog #3: eudyptula – challenge #1

Journalog #1: eudyptula

I have no idea if it’s still alive, but I want to participate in the eudyptula challenge again. Last time I did it was probably a little less than 2 years ago, and I was still playing with with Arch at the time.

I am using OS X this month, which screws the whole thing from the beginning. I have to use a virtual machine then. VMWare Fusion is already installed in my system, so whatever. I have to grab a Linux box now. Heck…something very lightweight, no-frills. First thing that comes to mind: CRUX. Yeah, it’s you baby.

ISO file is ~400MB in size. I said lightweight, not small. Installation process is cool.

Screen Shot 2016-07-12 at 9.08.22 PM5 min or so and everything is properly set up. Bootloader, fstab…nothing fancy. CRUX uses LILO (I forgot about that!), but this is a simulated BIOS system anyway (in VMWare), so TO /dev/null, mister GRUB2! And you then reach a few seconds of enlightenment, away from the 21st century. systemd? Xorg? What’s that? OK, there are some btrfs packages laying around. Still!! What? The folks at without-systemd.org (no hyperlink. Go figure.) have a point.

Serendipity: it turns out some people really take this seriously.

All set, rebooting. Let’s work!! I am going to do everything as root, I don’t care. Screw. It’s 2016, virtual machines do have snapshots, bae.

I need to send one plain text (non-HTML) email to start the challenge. That’s all. So I need to grab mutt. I don’t care about sendmail, mail, fetchmail, whatever; so I will need a TUI to do basic stuff. Yah, I know, mutt will use sendmail behind-the-scenes, whatever.

This leads me to…package management. How to install mutt? Heck, no apt-get, yum, pacman. I chose CRUX, now I pay for that, right?? Not really. CRUX has a very basic ports/package management system, I’d compare it to OpenBSD. Hopefully, mutt is going to be present there. It is. So…1 or 2 min reading its handbook and it turns out all I need is

ports -u && prt-get install mutt

Cool. Now here comes the PITA stuff. I have to get a very basic ~/.muttrc. Well, it would be a PITA if we were in the 2000s, but no one cares about man pages anymore(1). Stack Overflow and hippie blogs are the way to go. So a quick Googling here and there leads me to a basic IMAP + SMTP mutt setup with my email provider.

…er, no. Grrrrrr. Mutt does not recognize smtp. Whaaat?! It turns out the default mutt from CRUX is not compiled with smtp enabled. You gotta be kidding me. Okay, let’s compile it from source with the correct flags. I would usually probably take 20+ minutes to figure this out, but I think after all these years playing with linux distros I learned something. It took me less than 3 minutes to go to /usr/ports/opt/mutt and edit the Pkgfile over there, then run pkgmk and finally pkgadd -u to install it. I didn’t have any prior knowledge of this stuff, it was mostly a matter of Cmd-F’ing the handbook webpage and introspecting (=a fancy word for trial-and-error) the command line.

OK, mutt is now installed. WITH SUPPORT FOR EVERYTHING I NEED, damn it. Now…let’s log in. Cool, it works. Actually…my email provider (Google, yaaay) tells me I am logging in from an insecure app, and thus it blocks my login attempt. Aaaaaaaa….ok, I like this feature! But I temporarily disable it (note to self: re-enable it later, you security freak creature!).

Test #1: hello world email sent back to myself. OK, my phone rings. In fact, twice. Leave me alone, I am migrating my apps, I have more than one email client installed on it. You never did that?! Well…everything’s working, so I just send an email to little@eudyptyla.

And this is supposed to be a happy ending. For today. Except that…maybe this challenge does not even exist anymore. I hope it does, I really want to do it. Time is key.

Screen Shot 2016-07-12 at 9.35.04 PM.png

Bye.

(1) man anymore: this is my first footnote. For more information, see anymore(8).

Journalog #1: eudyptula