Visualize Mt St Helen with Python and a custom color palette

Evan Bianco of Agile Geoscience wrote a wonderful post on how to use python to import, manipulate, and display digital elevation data for Mt St Helens before and after the infamous 1980 eruption. He also calculated the difference between the two surfaces to calculate the volume that was lost because of the eruption to further showcase Python’s capabilities. I encourage readers to go through the extended version of the exercise by downloading his iPython Notebook and the two data files here and here.

I particularly like Evan’s final visualization (consisting of stacked before eruption, difference, and after eruption surfaces) which he created in Mayavi, a 3D data visualization module for Python. So much so that I am going to piggy back on his work, and show how to import a custom palette in Mayavi, and use it to color one of the surfaces.

Python Code

This first code block imports the linear Lightness palette. Please refer to my last post for instructions on where to download the file from.

import numpy as np
# load 256 RGB triplets in 0-1 range:
LinL = np.loadtxt('Linear_L_0-1.txt') 
# create r, g, and b 1D arrays:
r=LinL[:,0] 
g=LinL[:,1]
b=LinL[:,2]
# create R,G,B, and ALPHA 256*4 array in 0-255 range:
r255=np.array([floor(255*x) for x in r],dtype=np.int) 
g255=np.array([floor(255*x) for x in g],dtype=np.int)
b255=np.array([floor(255*x) for x in b],dtype=np.int)
a255=np.ones((256), dtype=np.int); a255 *= 255;
RGBA255=zip(r255,g255,b255,a255)

This code block imports the palette into Mayavi and uses it to color the Mt St Helens after the eruption surface. You will need to have run part of Evan’s code to get the data.

from mayavi import mlab
# create a figure with white background
mlab.figure(bgcolor=(1, 1, 1))
# create surface and passes it to variable surf
surf=mlab.surf(after, warp_scale=0.2)
# import palette
surf.module_manager.scalar_lut_manager.lut.table = RGBA255
# push updates to the figure
mlab.draw()
mlab.show()

After_Linear_L

Reference

Mayavi custom colormap example

 

Related Posts (MyCarta)

The rainbow is dead…long live the rainbow! – Part 5 – CIE Lab linear L* rainbow

Convert color palettes to Python Matplotlib colormaps

Related Posts (external)

How much rock was erupted from Mt St Helens

Convert color palettes to python matplotlib colormaps

This is a quick post to show you how to import my perceptual color palettes – or any other color palette – into Python and convert them into Matplotlib colormaps. We will use as an example the CIE Lab linear L* palette, which was my adaptation to Matlab of the luminance controlled colormap by Kindlmann et al.

Introduction

You can use the code snippets in here or download the iPython notebook from here (*** please see update ***). You will need NumPy in addition to Matplotlib.

***  UPDATE  ***

Fellow Pythonista Matt Hall of Agile Geoscience extended this work – see comment below to include more flexible import of the data and formatting routines, and code to reverse the colormap. Please use Matt’s expanded iPython notebook.

Preliminaries

First of all, get the color palettes in plain ASCII format rom this page. Download the .odt file for the RGB range 0-1 colors, change the file extension to .zip, and unzip it. Next, open the file Linear_L_0-1, which contains comma separated values, replace commas with tabs, and save as .txt file. That’s it, the rest is in Python.

Code snippets – importing data

Let’s bring in numpy and matplotlib:

%pylab inline
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt

Now we load the data. We get RGB triplets from tabs delimited text file. Values are expected in the 0-1 range, not 0-255 range.

LinL = np.loadtxt('Linear_L_0-1.txt')

A quick QC of the data:

LinL[:3]

gives us:

array
([[ 0.0143,  0.0143,  0.0143],
  [ 0.0404,  0.0125,  0.0325],
  [ 0.0516,  0.0154,  0.0443]])

looking good!

Code snippets – creating colormap

Now we want to go from an array of values for blue in this format:

b1=[0.0000,0.1670,0.3330,0.5000,0.6670,0.8330,1.0000]

to a list that has this format:

[(0.0000,1.0000,1.0000),
(0.1670,1.0000,1.0000),
(0.3330,1.0000,1.0000),
(0.5000,0.0000,0.0000),
(0.6670,0.0000,0.0000),
(0.8330,0.0000,0.0000),
(1.0000,0.0000,0.0000)]

# to a dictionary entry that has this format:

{
'blue': [
(0.0000,1.0000,1.0000),
(0.1670,1.0000,1.0000),
(0.3330,1.0000,1.0000),
(0.5000,0.0000,0.0000),
(0.6670,0.0000,0.0000),
(0.8330,0.0000,0.0000),
(1.0000,0.0000,0.0000)],
...
...
}

which is the format required to make the colormap using matplotlib.colors. Here’s the code:

b3=LinL[:,2] # value of blue at sample n
b2=LinL[:,2] # value of blue at sample n
b1=linspace(0,1,len(b2)) # position of sample n - ranges from 0 to 1

# setting up columns for list
g3=LinL[:,1]
g2=LinL[:,1]
g1=linspace(0,1,len(g2))

r3=LinL[:,0]
r2=LinL[:,0]
r1=linspace(0,1,len(r2))

# creating list
R=zip(r1,r2,r3)
G=zip(g1,g2,g3)
B=zip(b1,b2,b3)

# transposing list
RGB=zip(R,G,B)
rgb=zip(*RGB)
# print rgb

# creating dictionary
k=['red', 'green', 'blue']
LinearL=dict(zip(k,rgb)) # makes a dictionary from 2 lists

Code snippets – testing colormap

That’s it, with the next 3 lines we are done:

my_cmap = matplotlib.colors.LinearSegmentedColormap('my_colormap',LinearL)
pcolor(rand(10,10),cmap=my_cmap)
colorbar()

which gives you this result: LinearL test Notice that if the original file hadn’t had 256 samples, and you wanted a 256 sample color palette, you’d use the line below instead:

my_cmap = matplotlib.colors.LinearSegmentedColormap('my_colormap',LinearL,256)

Have fun!

Related Posts (MyCarta)

The rainbow is dead…long live the rainbow! – Part 5 – CIE Lab linear L* rainbow

Visualize Mt S Helens with Python and a custom color palette

A bunch of links on colors in Python

http://bl.ocks.org/mbostock/5577023

http://matplotlib.org/api/cm_api.html

http://matplotlib.org/api/colors_api.html

http://matplotlib.org/examples/color/colormaps_reference.html

http://matplotlib.org/api/colors_api.html#matplotlib.colors.ListedColormap

http://wiki.scipy.org/Cookbook/Matplotlib/Show_colormaps

http://stackoverflow.com/questions/21094288/convert-list-of-rgb-codes-to-matplotlib-colormap

http://stackoverflow.com/questions/16834861/create-own-colormap-using-matplotlib-and-plot-color-scale

http://stackoverflow.com/questions/11647261/create-a-colormap-with-white-centered-around-zero

NASA Worldview satellite image browser adopts MyCarta perceptual rainbow

I was thrilled this week to learn from Ryan Boller that his team at NASA’s ESDIS Project included MyCarta’s perceptual rainbow (the CubicYF) as one of the palettes for the Worldview satellite imagery browser.

If you’d like to try it, once on the viewer you can load an overlay and then you can choose from among several color palettes. The perceptual rainbow palette is listed here as “Rainbow 2”.

I am including below an example using the Land surface temperature for April 13 2013 from MODIS Aqua mission:

Land_surf_temp_130413

This is really exciting news as NASA’s adoption will increase the palette’s exposure and its chances of becoming more mainstream. This is also as close as I will ever get to realizing my childhood dream of becoming an astronaut. Thanks ESDIS, and thanks Ryan, on both accounts.

An introduction to color palettes for seismic amplitude – teaser

Introduction

In a future posts I will take a look at some of the color palettes used for seismic amplitude display, and discuss ways we can design more perceptual and more efficient ones.

For now, I would like to ask readers to look at two sets of seismic images and answer the survey questions in each section. Far from being exhaustive sets, these are meant as a teaser to get a conversation started and exchange opinions and preferences.

Stratigraphic interpretation

The seismic line below is inline 424 from the F3 dataset, offshore Netherlands from the Open Seismic Repository (licensed CC-BY-SA).

I generated an animation, played at 0.5 frames/second, where 8 different color palette are alternated in sequence.  Please click on the image to see a full resolution animation. I also generated a 0.25 frame/second version and a 1 frame/second version.

05

Fault interpretation

The images used to create the panel below are portions of seismic displays kindly provided by Steve Lynch of 3rd Science Solutions, generated using data released by PeruPetro. I am grateful to both.

faults_sm

Thanks to Matt Hall and Evan Bianco of Agile Geoscience for their suggestions.

Color palettes for seismic structure maps and attributes

I created three color palettes for structure maps (seismic horizons, elevation maps, etcetera) and seismic attributes. To read about the palettes please check these previous blog posts:

The rainbow is dead…long live the rainbow! – Part 5 – CIE Lab linear L* rainbow
The rainbow is dead series – Part 7 – Perceptual rainbow palette – the method
The rainbow is dead series – Part 7 – Perceptual rainbow palette – the goodies

The palettes are available as plain ASCII files and also formatted for a number of platforms and software products:

Geosoft
Hampson-Russell
Kingdom
Madagascar
Matlab
OpendTect
Petrel
Seisware
Surfer
Voxelgeo

Please download them from my Color Palettes page and follow instructions therein.

Enjoy!

linearlfb

Image courtesy of Sergey Fomel of the Madagascar Development blog

Perceptual rainbow palette – the goodies

Perceptual rainbow palette – Matlab function and ASCII files

In my last post I introduced cubeYF, my custom-made perceptual lightness rainbow palette. As promised there, I am sharing the palette  with today’s post. For the Matlab users, cube YF, along with the other palettes I introduced in the series, is part of the Matlab File Exchange submission Perceptually improved colormaps.

For the non-Matlab users, please download the cubeYF here (RGB, 256 samples). You may also be interested in cube1, which has a slightly superior visual hue contrast, due to the addition of a red-like color at the high lightness end but at the cost of a modest deviation from 100% perceptual. I used cube 1 in my Visualization tips for geoscientists series.

Perceptual rainbow palette – preformatted in various software formats

The palettes are also formatted for a number of platforms and software products: Geosoft, Hampson-Russell, SMT Kingdom, Landmark Decision Space Geoscience, Madagascar, OpendTect, Python/Matplotlib, Schlumberger Petrel, Seisware, Golden Software Surfer, Paradigm Voxelgeo. Please download them from my Color Palettes page and follow instructions therein.

Another example

In Comparing color palettes I used a map of South America [1] to compare a linear lightness palette to some common rainbow palettes using  grayscale as a perceptual benchmark. Below, I am doing the same for the cubeYF colormap.

South_America_maps_CubeYF_rainbow

Comparison of South America maps using, from left to right: ROYGBIV (from this post) , classic rainbow, cubeYF, and grayscale

Again, there is little doubt in my mind that cubeYF does a superior job compared to the other two rainbow palettes as it is free of artefacts [2] and more similar to grayscale  (with the additional benefit of color).

The ROYGBIV and cubeYF map have been included in Marek Kultys’ excellent tutorial Visual Alpha-Beta-Gamma: Rudiments of Visual Design for Data Explorers, recently published  on Parsons Journal for information mapping, Volume V, Issue 1.

An online palette testing tool

Both cubeYF and cube1 feature in the colormap evaluation tool by the Data Analysis and Assessment Center at the Engineer Research and Development Center. If you want to quickly evaluate a number of palettes, this is the right tool. The tool has a collection of many palettes, organized by categories, which can be used on 5 different test image, and examined in terms of RGB components and human perception. Below here is an example using cube YF.

hpc_terrain

An idea for a palette’s mood test

A few weeks ago, thanks to Matt Hall (@kwinkunks on twitter),  I discovered Colour monitor, a great online tool by Richard Weeler (@Zephyris on twitter). You supply an image; Colour monitor analyses its colors in terms of hue, saturation and luminance and produces a graphical representation of the image’s mood [3]. I thought, what a wonderful idea!

Then I wondered: what if I used this to tell me something about a color palette’s mood? The circular histogram of colors reminded me of the Harmonic templates [4] on the hue wheel from this paper And so I created fat colorbars using the three  palettes I used in the last post, saved them as images, and run the monitor with them. Here below are the results for Matlab jet, Industry Spectrum, and cubeYF. Looking at these palettes in terms of harmony I would say that jet is not very harmonic (too large a portion of the hue circle; the T template, which is the largest, spans 180 degrees), and that the spectrum is terrible.

CubeYF is also exceeding a bit 180 degrees, but looks very close to a T template rotated by 180 degrees (rotations are allowed). So perhaps I could trim it a bit? But to me it looks a lot nicer and gives me a vibe of really good mood, and reminds me of one of those beautiful central american headdresses, like Moctezuma’s crown.

jet-clrmp-mood

Jet mood

Spectrum-industry-clrmp-mood

Spectrum mood

cubeYF-clrmp-mood

cubeYF mood

Notes

[1] Created with data from the Global Land One-km Base Elevation Project at the National Geophysical Data Center.

[2] Looking at the intensity of the colorbars may help in the assessment: the third and fourth colorbars are very similar and both look perceptually linear, whereas the first and second do not.

[3] Quoted from Richard’s blog post: “… in the middle is a circular histogram of the colours (spectral shades) in the image, and gives an idea of how much of each colour there is. Up the left is a histogram of image brightness (lightness of colour), and up the right is a histogram of colour saturation (vibrancy)”.

[4] Quoted from the paper’s abstract: “Harmonic colors are sets of colors that are aesthetically pleasing in terms of human visual perception. If you are interested in this idea there is a set of slides and a video on the author’s website

Related posts

Perceptual rainbow palette – the method

With this post I would like to introduce my new, perceptually balanced rainbow color palette. I used the palette for the first time in How to assess a colourmap, an essay I wrote for 52 Things You Should Know About Geophysics, edited by Matt Hall and Evan Bianco of Agile Geoscience.

In my essay I started with the analysis of the spectrum color palette, the default  in some seismic interpretation softwares, using my Lightness L* profile plot and Great Pyramid of Giza test surface (see this post for background on the tests and to download the Matlab code). The profile and the pyramid are shown in the top left image and top right image in Figure 1, from the essay.

spectrum vs cubeYF

Figure 1

In the plot the value of L* varies with the color of each sample in the spectrum, and the line is colored accordingly. This erratic profile highlights several issues with spectrum: firstly, the change in lightness is not monotonic. For example it increases from black (L*=0) to magenta [M] then drops from magenta to blue [B], then increases again and so on. This is troublesome if spectrum is used to map elevation because it will interfere with the correct perception of relief, particularly if shading is added. Additionally, the curve gradient changes many times, indicating a nonuniform perceptual distance between samples. There are also plateaus of nearly flat L*, creating bands of constant color (a small one at the blue, and a large one at the green [G]).

The Great Pyramid has monotonically increasing elevation (in feet – easier to code) so there should be no discontinuities in the surface if the color palette is perceptual. However, clearly using the spectrum we have introduced many artificial discontinuities that are not present in the data. For the bottom row in FIgure 1 I used my new color palette, which has a nice, monotonic, compressive Lightness profile (bottom left). Using this palette the pyramid surface (bottom right) is smoothly colored, without any perceptual artifact.

This is how I created the palette: I started with RGB triplets for magenta, blue, cyan, green, and yellow (no red), which I converted to L*a*b* triplets using Colorspace transformations, a Matlab function available on the Matlab File Exchange. I modified the new L* values by fitting them to an approximately cube law L* function (this is consistent with Stevens’ power law of perception), and adjusted a* and b* values using Lab charts like the one in Figure 2 (from CIELab Color Space by Gernot Hoffmann, Department of Mechanical Engineering, University of Emden)  to get 5 colors moving up the L* axis along an imaginary spiral (I actually used tracing paper). Then I interpolated to 256 samples using the same ~cube law, and finally reconverted to RGB [1].

L*50_RGBval

Figure 2

There was quite a bit of trial and error involved, but I am very happy with the results. In the animations below I compare the spectrum and the new palette, which I call cubeYF, as seen in CIELab color space. I generated these animations with the method described in this post, using the 3D color inspector plugin in ImageJ:

I also added Matlab’s default Jet rainbow – a reminder that defaults may be a necessity, but in many instances not the ideal choice:

OK, the new palette looks promising, insofar as modelling is concerned. But how would it fare using some real data? To answer this question I used a residual gravity map from my unpublished thesis in Geology at the University of Rome. I introduced this map and discussed the geological context and objectives of the geophysical study in a previous post, so please refer to that if you are curious about it. In this post I will go straight to the comparison of the color palettes; if you are unfamiliar with gravity data, try to imagine negative residuals as elevation below sea level, and positive residuals as elevation above seal level – you won’t miss out on anything.

In Figures 3 to 6 I colored the data using the above three color palettes, and grayscale as benchmark. I generated these figures using Matlab code I shared in my post Visualization tips for geoscientists: Matlab, and I presented three of them (grayscale, Spectrum, and cubeYF) at the 2012 convention of the Canadian Society of Exploration Geophysicists in Calgary (the extended abstract, which I co-authored with Steve Lynch of 3rd Science, is available here).

In Figure 3, the benchmark for the following figures, I use grayscale to represent the data, assigning increasing intensity from most negative gravity residuals in black to most positive residuals in white (as labeled next to the colorbar). Then, I used terrain slope to create shading: the higher the slope, the darker the shading that is assigned, which results in a pseudo-3D display that is very effective (please refer to Visualization tips for geoscientists: Surfer, for an explanation of the method, and Visualization tips for geoscientists: Matlab for code).

Figure 3 - Grayscale benchmark

Figure 3 – Grayscale benchmark

In Figure 4 I color the pseudo-3D surface with the cubeYF rainbow. Using this color palette instead of grayscale allows viewers to appreciate smaller changes, more quickly assess differences, or conversely identify areas of similar anomaly, while at the same time preserving the peudo-3D effect. Now compare Figure 4 with Figure 5, where we use the spectrum to color the surface: this palette introduces several artefacts (sharp edges and bands of constant hue) which confuse the display and interfere with the perception of pseudo-relief, all but eliminating the effect.  For Figure 6 I used Matlab’s default Jet color palette, which is better that the spectrum, and yet the relief effect is somewhat lost (due mainly to a sharp yellow edge and cyan band).

campi cube YF

Figure 4 – cube YF rainbow

campi spectrum

Figure 5 – Industry spectrum

campi jet

Figure 6 – Matlab Jet

It looks like both spectrum and jet are poor choices when used for color representation of a surface, with the new color palette a far superior alternative. In the CSEG convention paper mentioned above (available here) Steve and I went further by showing that the spectrum not only has these perceptual artifacts and edges, but it is also very confusing for viewers with deficient color vision, a condition that occurs in about 8% of Caucasian males. We did that using computer software [2] to simulate how viewers with two types of deficient color vision, Deuteranopia and Tritanopia, would see the two colored surfaces, and we compare the results. In other words, we are now able to see the images as they would see them. Please refer to the paper for a full discussion on these simulation.

In here, I show in Figures  7 to 9 the Deuteranope simulations for cubeYF, spectrum, and jet, respectively. In all three simulations the hue discrimination has decreased, but while the spectrum and jet are now even more confusing, the cubeYF has preserved the relief effect.

Deuteranope Simulation of campi cube YF

Deuteranope Simulation of cube YF

Deuteranope Simulation of campi spectrum

Deuteranope Simulation of Industry spectrum

Deuteranope Simulation of campi jet

Deuteranope Simulation of Matlab Jet

That’s it for today. In my next post, to be published very shortly, you will get the palette, and a lot more.

References

A more perceptual color palette for structure maps, CSEG/CSPG 2012 convention, Calgary

How to assess a colourmap, in 52 Things You Should Know About Geophysics

Notes

[1] An alternative to the method I used would be to start directly in CIELab color space, and use a some kind of spiral *L lightness profile programmatically.  For example:

– Using 3D helical curves from: http://www.mathworks.com/matlabcentral/fileexchange/25177-3d-curves

– Using Archimedes spiral

– Expanding on code by Steve Eddins at Mathworks (A path through L*a*b* color space) in this article , one could create a spiral cube lightness with something like:

%% this creates best-fit pure power law function 
%  Inspired by wikipedia - http://en.wikipedia.org/wiki/Lightness
l2=linspace(1,power(100,0.42),256); 
L2=(power(l2,1/0.42))'; 

%% this makes cielab real cube function spiral 
radius = 50; 
theta = linspace(0.6*pi, 2*pi, 256).'; 
a = radius * sin(theta); b = radius * cos(theta); 
Lab1 = [L2, a, b]; RGB_realcube=colorspace('RGB<-Lab',(Lab1));

[2] The simulations are created using ImageJ, an open source image manipulation program, and the Vischeck plug-in. I later discovered Dichromacy, anther ImageJ plug-in for these simulations, which has the advantage of being an open source plugin. They can also be performed on the fly (no upload needed) using the online tool Color Oracle.

Related posts

Comparing color palettes

Introduction

In my last post I introduced a CIE Lab linear L* rainbow palette from a paper by Kindlmann et al. [1]. I used this palette with a map of South America created with data from the Global Land One-km Base Elevation Project at the National Geophysical Data Center. The map is the third one in the figure below.

South_America_maps_LinearL_rainbow

Based on visual inspection I argued that linear L* colored map compares more favourably with the grayscale – my perceptual benchmark – on the right – than the first and second, which use my ROYGBIV rainbow palette (from this post) and a classic rainbow palette, respectively. I noted that looking at the intensity of the colorbars may help in the assessment: the third and fourth colorbars are very similar and both look perceptually linear, whereas the first and second do not.

So it seems that among the three color palettes the third ones is the best, but…..

… prove it!

All the above is fine and reasonable, and yet it is still very much subjective. How can I prove it, convince myself this is indeed the case?

Well, of course one way is to use my L* profile and Great Pyramid tests with Matlab code from the first post of this series. Look at the two figures below: comparison of the lightness L* plots clearly shows the linear L* palette is far more perceptual than the ROYGBIV.

L plot linear L L plot ROYGBIV

One disadvantage of this method is that you have to use Matlab, which is neither free nor cheap, and have to be comfortable with some code and ASCII file manipulation.

Just recently I had an idea for an open source alternative with ImageJ and the 3D color inspector plugin. The only preparatory step required is to save a palette colorbar as a raster image. Then open the image in ImageJ, run the plugin and display the colorbar in Lab space in a 3D view. There are many options to change the scale of the plot, the perspective, and how the colors are displayed (e.g. frequency weighted, median cut, etcetera). The view can be rotated manually, and also automatically.  Below I am showing the rotating animations for the same two palettes.

Discussion

The whole process, including the recording of the animations using  the Quicktime screencast feature, took me less than 10 minutes, and it leaves no doubt as to which one is the best color palette. Let me know what you think.

A few observations: in 3D the ROYGBIV palette is even more strikingly and obviously non-monotonic. The lightness gradient varies in magnitude, resulting in non-uniform contrast. Compare for example the portion between blue and green to that between green and yellow: these have approximately the same number of samples but very different change in lightness value between the extremes. The gradient sign also changes, producing perceptual inversions, for example with the yellow to red section following the blue to yellow. These inversions may result in perceived elevation inversions, for example, if using this palette to display elevation data. On the other hand, the linear L* palette nicely spirals upwards with L* changing monotonically from 0 to 100.

References

[1] Kindlmann, G. Reinhard, E. and Creem, S., 2002, Face-based Luminance Matching for Perceptual Colormap Generation, IEEE – Proceedings of the conference on Visualization ’02

Related posts (MyCarta)

The rainbow is dead…long live the rainbow! – the full series

What is a colour space? reblogged from Colour Chat

Color Use Guidelines for Mapping and Visualization

A rainbow for everyone

Is Indigo really a colour of the rainbow?

Why is the hue circle circular at all?

A good divergent color palette for Matlab

Related topics (external)

Color in scientific visualization

The dangers of default disdain

Color tools

How to avoid equidistant HSV colors

Non-uniform gradient creator

Colormap tool

Color Oracle – color vision deficiency simulation – stand alone (Window, Mac and Linux)

Dichromacy –  color vision deficiency simulation – open source plugin for ImageJ

Vischeck – color vision deficiency simulation – plugin for ImageJ and Photoshop (Windows and Linux)

For teachers

NASA’s teaching resources for grades 6-9: What’s the Frequency, Roy G. Biv?

ImageJ and 3D Color inspector plugin

http://rsbweb.nih.gov/ij/docs/concepts.html

http://rsb.info.nih.gov/ij/plugins/color-inspector.html