Saturday, July 15, 2017

My developer toolkit 2017 (Mac)

Back in December 2010 I had blogged about the powertools I was using with windows. Soon I will do a revamp of the Windows powertools which I am using on my Windows 10 PC.This post is about the list of tools that I use for my day today activities on my Mac.

Terminal Utilities

Compared to the default terminal, I prefer to use iTem2. iTerm also integrates with Oh My Zsh which makes life lot more easier while working with terminal.Refer to the Oh my Zsh cheatsheet for more details. I love all the aliases related to Git like gst for git status, gcmsg for git commit –m and many other git commands. I use the Avit theme with Oh My Zsh which gives nice look and feel to the different commands and their outputs on the terminal.

While working with tabbed windows on the terminal it can be quite confusing and hard to remember what you were doing with which terminal window. Tabset helps to name the tabs and also give different colors to them. Display of the tab name on the right hand corner is quite helpful for me.

Code Editors

The more I use VS Code, more I am liking the features of it. It is very elegant and has a nice Themes and Plugins based ecosystem for enhancing its capabilities. No wonder more and more people (even those who hated Microsoft) are using VS Code. If you don’t want the full featured IDE of Visual Studio 2015 / 2017 but still need the better part of code editing go for VS Code.

Before I started using VS Code, I was a fan of Atom. I like its simplicity. It also integrates very well with the GitHub (it is actually created by Github). They call it the hacakble text editor for 21st century. Many people complain about slowness of Atom editor. I did not face any issues so far. May be the files I was dealing with were within the bounds of Atom.

When I moved from Windows 10 to Mac, I started using Sublime Text 3. I find it similar to Atom in many ways. Sublime is the oldest editor among VS Code, Atom & Sublime. As a result it has more features, themes and plugins.

When working with Java, I use IntelliJ Idea from Jetbrains. It is one of the best IDE I have come across (obviously after Visual Studio). I find IntelliJ much more easier to adapt coming from .Net world as compared to Eclipse.  The dark theme of IntelliJ makes me feel at home.

One of the plugin which I find very helpful in IntelliJ is the Key Promoter. It tells you how many times you have used mouse when there is a keyboard shortcut available for a command. I feel this is really needed for all the developers if you want to get better at keyboard shortcuts.

Although I prefer to work with terminal while using Git & Github repositories, I find the GitHub desktop tool handy when I want to do some GUI related work.

All the 3 text editors seem to have quite a few things in common. Especially plugins and themes are mostly ported from one editor to the other. I like the Material Theme and Monokai. Best part is all 3 editors are cross platform and work with Windows and Mac. That definitely reduces the learning curve.

Virtualization software

Docker allows to spin up lightweight containers as compared to full blown virtual images. If you don’t want to mess around with your laptop but want to try out some new tool, Docker is a good way to test it.

Not all things can be done via Docker containers.Sometimes you still need to use a virtual image. I tend to use VirtualBox for such cases.

Vagrant allows to configure virtual machines in a declarative manner. It uses VirtualBox as the provider for creating virtual machines.

General utilities

I use Ansible to automate the installation of software as much as possible on my Mac Book Pro. Refer to my post on Setup Mackbook almost at  the speed of light to know about it in more details

This is my favorite notes taking app. Works across all my devices including iPhone, iPad Pro, Mac book and windows PC. I like the simplicity of the tool. It resembles the physical notebook. The organization of notes into workbooks and pages is something I like very much when it comes to notes taking. I tried other alternatives like Evernote.

I use KeePass on Windows 10. There is a nice port of it available on Mac called MacPass. Use it if you want to store all your passwords in one place.

Todoist is one of my favourite task organizer. It works cross platform and has support for mobile devices as well. it is very simple to use and has a minimalistic UI. I have tried others like Evernote but prefer Todoist for its simplicity.

Dropbox is my preferred way for synching documents across devices. I also use Google Drive and Microsoft One Drive for different documents.

On Windows, I am a big fan of Open Live Writer for writing blog. Unfortunately it works only with Windows. On Mac I found Blogo. Although it is not as feature rich as Live Writer it solves the purpose.

I mostly read ebooks in PDF format. Acrobat reader allows synching of ebooks across devices using Adobe Cloud.

CheatSheet is one of the best free utility I have ever come across. It displays all the keyboard shortcuts for any application that you are currently running. You don’t need to remember each and every shortcut. Just hold the hotkey (by default the command key) for CheatSheet and you will see all the relevant keyboard shortcuts. I have started holding the Alt key on windows keyboard hoping to see keyboard shortcuts when I work on Windows 10 nowadays Smile

Spectacle is another nice little utility which allows you to resize & position the application windows with keyboard shortcuts. I also have a secondary monitor attached to my laptop. Spectacle is very helpful in moving windows across screens. Even if you don’t have multiple screens, you can still use spectacle to great effect to resize and position the windows.

f.lux is a utility which works with both Windows and Mac. It automatically adjusts the brightness of screen based on the time of the day.

Battery Related utilities

Has a very simplistic UI. Provides notifications on the battery levels. I find it useful to be notified when the battery is fully charged.

There are couple of other battery related apps that I am testing currently before picking up the one for my needs. These include coconut battery, Battery Heath, Battery Guardian.

Conclusion

The tolls that we use keep changing every year. I am sure there will be many more tools and utilities out there which would help to make our life more simpler and easier to work with machines. I would be interested in knowing such tools.

Tuesday, July 11, 2017

The ‘Yes’ command

Background

Recently I was working on a personal project involving GitHub and Travis CI. This short post is about my experience of hacking some of the options with Travis CI. To start with, Travis CI is giving free access to all public repositories on GitHub to run continuous integration. I used this to setup CI build for my Mac Dev Setup project. Once again, the initial work of Travis build definition has been done by Jeff Geerling. I am extending his goodwork to incorporate my changes.

One of the best part of Travis CI is the option to chose as Mac OS to run the CI build. You can refer to link for more details. I started off with the xcode7.3 version of the image. This is the default image at the time of writing this blog. The build was working fine with this version of them image. So I thought of upgrading the next image version of OSX with label xcode8. This build was successful without any changes.

Problem with Homebrew uninstallation

I thought that it was quite easy to just change the image version and the builds would work without any problem. Unfortunately not. I skipped the xcodee8.1 & xcode8.2 versions and tried to jump to xcode8.3 directly. The build failed with timeout. Looking at the build log, I can find out that the build was waiting for the confirmation on removal of Homebrew package and was expecting an input to the prompt in the form of y/N. Look at line number 391 in the below screenshot.

So I thought of downgrading the image version to 8.2. It was still the same. Hmm. Something had changed between the 8 version and the others. So I reverted back to the least version after 8 which was 8.1. As part of the initial setup, I was uninstalling the Homebrew package and from 8.1 image version, the installer expects a confirmation. Not sure why it doesn’t do it in the earlier versions.

I was running a script by downloading it and running a Ruby command as
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall"

Ruby does not provide an option to pass default as ‘Y’ to any of the prompts. Atleast I did not find such option using my google search skills. I started looking for ways to silently invoke the command and pass ‘Y’ as the default answer to the prompt.

‘Yes' command to the rescue

There were multiple solutions available online. But I liked the one provided by a command strangly named as yes. It can provide input as y or n to any prompt. The pipelining of commands and utilities in Unix / Linux based system helped here to pipe the yes command with the ruby script which I was using. The final command looks like
yes | ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall"

Note that default is Y for the yes command. If you wish to answer n, you can change the syntax to yes n :). It is quite ironic to use yes command and reply as no. But thats how the author of this command desired it. They could have create a complimentary no command which would respond with n. I even did a google search to check if there is such command. Unforunately it does not exist.

With the help of wonderful yes comand my build is running fine now. I don’t know if there is any better way of supplying an answer to the prompt on Travis CI. If you know lt me know via comments.

Monday, July 03, 2017

Setup MacBook almost at the speed of light

Background

I bought a new MacBook recently. It is always fascinating to setup your new machine. But it is also a pain to look for all the tools that you have on your old machine and port it to the new machine. Sometime back I started learning abount Ansible which helps to automate routine tasks. I came across a blog by Jeff Geerling who is the author of book Ansible for DevOps. Jeff and many others had used Ansible to setup their machines. I took inspiration from these guys blogs to automate the process of setting up a new MacBook Pro. Here is my experience.

Why Ansible?

Ansible is very easy to understand. It uses human readbale YAML syntax to describe the different actions which needs to be performed. Group of Ansible actions which are executed as part of a playbook are idempotent. It does not have a side effect on the setup. The same playbook can be run multiple times. Only the changes will be applied incrementally.


How did I use Ansible?

I started off by cloning the Git repository of Geerlingguy which is a good starting point. Jeff Geerling has done a very good job in terms of laying out the framework for initial set of tools. Jeff used Homebrew as package manager for installing packages. For the UI applications Homebrew Cask is used.I added some of the applications which were not exising in the original repo of Jeff Geerling.

It is very easy to get started. The repo follows the best practices from Ansible world and organizes the different topics into structure shown in th image below

Lets start with looking at some of the important files & folders from this repo.

The files directory contains additional files required for configuring specific tools. Jeff Geerling had custom options / configurations for Sublime and terminal. I added zshrc.in. The zshrc.in file is the dotfile for Oh My Zsh. We will talk about Oh My Zsh a bit later in this post.

Roles directory contains the Ansible Roles required for executing different tasks as part of the playbook. Here again, from the original repo of Jeff Geerling there were roles for managing dot files, home-brew, mas and command line tools. I added the role for managing atom packages.

Tasks folder contains list of tasks or actions which needs to performed during installation. These are organized into multiple files like ansible-setup, extra-packages-setup etc. I added a file for oh-my-zsh-setup.

The important files in the complete structure are default.config.yml and main.yml. The main.yml file is the glue that binds all the things together. This is like a main program in programming language like C# or Java. It contains references to the runtime variables, roles used and the order in which the tasks needs to be executed.
The default.config.yml file contains all the variables used by the tasks. It contains the list of tools & applications to be installed or uninstalled as part of the playbook. One of the advantage of using this approach is the applications which are installed gets moved to Applications folder as part of tasks. If we install the applications manually, sometimes we need to move them from downloads or other folder to Applications.

Apart from the applications itself, I also needed some additional libraries / tools. There were some which I was not using so I deleted those packages. Below are some of the additions / enhancements I did to meet my needs.

I made 2 major modifications to the repo of Jeff Geerling. I added the automatic configuration of Oh My Zsh and also Atom plugins & themes. Below steps were needed to make these modifications.

Setup Oh My Zsh

I like to use the Oh My Zsh as it enhances the default terminal with a better experience. It uses zsh as an alternative terminal to default terminal. Oh My ZSH is community driven framework for managing zsh configurations. It has lots of Themes & Plugins support and makes working on the terminal a really enjoyable experience. Doing a bit of Google search brought me to the GitHub repo for setting up Oh My Zsh by Remy Van Elst. I copied the zshrc.in file in the files directory. Same way I added the oh-my-zsh-setup.yml file in the tasks directory. The last step was to add an include statement to the main.yml file file to include oh-my-zsh-setup.yml file in the tasks definition.

Setup Atom plugins

Over the last few months, I had been using Atom as text editor. I used multiple Atom plugins and Themes. Atom has very good support for installing plugins using command line options. I especially like the Material UI theme which is supported by multiple editors including Atom & Sublime. I really like the minimilatic design of Atom editor.
It would be nice to have these plugins also installed as part of the machine setup. Fortunately there is an Ansible role hy Hiroaki Nakamura which allows exactly this functionality. You provide a list of Atom Themes & Plugins to this role and your machine will have all of them installed using Ansible. This is awesome. No need to go & search for plugins in the Atom UI. After the initial set of plugins, I have used the playbook for adding new ones with effortless ease.

To use the role, I added the role definition to the requirements.yml file. This file contains the list of roles whic need to be downloaded. As a pre-condition, all the roles listed here are downloaded before running any tasks. The hnakamur.atom-packages role expects a variable named atom_packages_packages. There is no difference between Themes & Plugins. I listed down all the Atom plugins & themes here. The last step was to include this role in the main.yml file.

Setup Visual Studio Code plugins & Themes

I have just started using the Visual Studio code editor. I was able to install VS Code using the default apps method from Jeff Geerling's playbook task. Similar to Atom or Sublime, VS Code has a rich support for Plugins & Themes. I found an Ansible Role by Gantsign named ansible-role-visual-studio-code. Looking at the readme file it seems to be made for Ubuntu. The role also installs VS Code editor. In my case I already have the editor installed using Homebrew Cask. I needed just the ability to install the plugins & themes.
From the code available within the repo, I found the code which is required to install the VS extension. The above role does a good job in installing VS code & extensions for multiple users of the system. Mine is a single user laptop & I did not need such functionality.

I ended up creating a file in the tasks named visual-studio-code-extensions-setup.yml. This file contains only one task of installing the extensions. The task wraps the command “code --install-extension extensionName”. The extension name is a placeholder in the above command and needs to be dynamically built. The default.config.yml defines a list of extensions in a variable named visual_studio_code_extensions. The extension name uses a specific format and it took me sometime to get hang of it. If we install the extension using VS Code IDE it works perfectly fine with just the extension name. But when we try to install the same extension using commandline, we need to prefix the publisher name. For e.g. the csharp extension is published by Microsoft. We need to provide the fully qualified name as ms-vscode.csharp.

The list of extensions can be specified as

visual_studio_code_extensions:
- steoates.autoimport
- PeterJausovec.vscode-docker
- ms-vscode.csharp

But this looks very clumsy. This is where the simplicity and flexibility of Ansible & YAML can be beneficial. We can define custom lists or dictionaries which can split the publisher & the extension name. I used this approach to define the extensions as
visual_studio_code_extensions:
- extensionName: autoimport
publisher: steoates
- extensionName: vscode-docker
publisher: PeterJausovec
- extensionName: csharp
publisher: ms-vscode

The tasks file then concatenates the publisher & the extension.

Next steps

There are still some manual steps involved in setting up the machine. As of now, I did not find a way to use Ansible to install applications from Apple App Store. I had to manually install one app Blogo which I used for writing this blog post. I am still looking for ways to automate this. There might be a way to invoke a shell command using Ansible which might allow to install App store apps. I have not tried it so far. Better way in my opinion would be to have a Ansible Role which can take a list of Apps to be installed and silently install them.

[Update]: I received a tweet from Jeff Geerling that there is mas role defined within his playbook which can be used to install apps from App Store by specifying the email & password linked to the Apple Id account. I will try this approach and update the contents accordingly.


Conclusion

At the end of the exercise, my laptop was setup and looked like below

All the applications that you see as well as additional ones which are not visible in this screen (like VS Code) were installed using Ansible playbook. It took just one single command to get these apps installed. The setup can be replicated to any other MacBook with minimal changes. Automating the installation steps have saved me much more time to do useful stuff (like writing this blog :)). You can also setup your Mac using similar steps. If you wish to do so refer to the readme file available at my Github repo. I intend to keep updating this repo with the changes that I am making to my dev environment. Feel free to contribute to this repo.

VSTS CI Build for Dockerized .net Core 2.1 multi-container app

Background This is 2nd part of the series on Continuous Deployment of Multi-Container app in Azure using VSTS and Docker . In the first part...