Image Alignment#
- Date:
2022-10-12
1 Giotto and Image Data#
Multiple images may be attached to the Giotto object. Spatial data can be overlaid upon these images which may then be used for downstream analyses. While Giotto attempts to automate the addition and alignment of images as much as possible, manual adjustments may sometimes be necessary. This tutorial will be covering both automatic and manual adjustment.
The examples in this tutorial will be worked using Visium’s normal human prostate FFPE dataset for both the Visium (automatic) and manual adjustment workflows. A manual download of this data is required; please see Visium Directory Structure Details below.
For an example of working with high definition images that may require ROI alignment and or stitching, please see the Nanostring CosMx lung data analysis.
# Ensure Giotto Suite is installed.
if(!"Giotto" %in% installed.packages()) {
devtools::install_github("drieslab/Giotto@suite")
}
# Ensure GiottoData, a small, helper module for tutorials, is installed.
if(!"GiottoData" %in% installed.packages()) {
devtools::install_github("drieslab/GiottoData")
}
library(Giotto)
# Ensure the Python environment for Giotto has been installed.
genv_exists = checkGiottoEnvironment()
if(!genv_exists){
# The following command need only be run once to install the Giotto environment.
installGiottoEnvironment()
}
2 Conceptual Overview#
Giotto currently supports two types of image objects. For most purposes, images are loaded in using terra and are then placed into giottoLargeImage objects. giottoLargeImage objects only load in sampled subsets of the original image that are needed to represent the image, increasing efficiency for analysis of very high-resolution images.
Alternatively, smaller images are loaded in using the magick package and are then placed into giottoImage objects. This offers direct access to the powerful image processing functionality in the magick package among other Giotto functions for analysis and viewing. giottoLargeImage objects can be downsampled into giottoImage objects when necessary through use of the convertGiottoLargeImageToMG() function.
More about the giottoImage container
giottoImages are S4 class objects with slots containing the image itself and the metadata necessary to plot it properly. The magick package allows easy access to image processing functions as well as the ability to access images through pointers, so the images are only loaded into memory when needed.
Please note that as a result of using pointers to access the image, giottoImages will not be saved between sessions, since the pointer will die after a given R session is closed. However, the information used to create the image will be retained within the giottoObject in the images slot, as described below.
*Note that minmax refers to the relevant values of the associated spatial locations rather than those of the image. These values are given either by providing spatial locations directly when calling createGiottoImage() or in later steps that involve a giottoObjects with associated spatial locations.
For instance, assume a giottoImage object named GImage has already been created:
GImage
## R TERMINAL OUTPUT:
#
# An object of class ' giottoImage ' with name image
#
# Min and max values are:
# Max on x-axis: 23520
# Min on x-axis: 5066
# Max on y-axis: -3682
# Min on y-axis: -23148
#
# Boundary adjustment are:
# Max adjustment on x-axis: 3949.001
# Min adjustment on x-axis: 5066
# Max adjustment on y-axis: 3682
# Min adjustment on y-axis: 2082.277
#
# Boundaries are:
# Image x-axis max boundary: 27469
# Image x-axis min boundary: 0
# Image y-axis max boundary: 0
# Image y-axis min boundary: -25230.28
#
# Scale factor:
# x y
# 0.07280935 0.07280935
#
# Resolution:
# x y
# 13.7345 13.7345
#
# File Path:
# [1] "/path/to/directory/tissue_image.png"
Further intuition for defining these parameters in this way is detailed within the Why this inversion is necessary dropdown text beneath Standard workflow.
3 Visium Workflow (Automated):#
Assembly of Giotto object as well as the reading in and alignment of the tissue staining image from the Visium spatial subdirectory is done automatically using createGiottoVisiumObject().
Note that in order to run the following code, the Output Files “Feature / barcode matrix (raw)” and “Spatial imaging data” from the dataset must be downloaded and extracted into a structured Visium directory.
Visium Directory Structure Details
Here, details on how to structure the Visium Directory for creating a Giotto object using createGiottoVisiumObject() for the purposes of this tutorial will be shown. Nonetheless, this procedure is standard practice for using Giotto with Visium Data.
First create a new directory. This will be the Visium Directory. Then, open a terminal within that directory, and enter the following commands:
wget https://cf.10xgenomics.com/samples/spatial-exp/1.3.0/Visium_FFPE_Human_Normal_Prostate/Visium_FFPE_Human_Normal_Prostate_raw_feature_bc_matrix.tar.gz
tar -xzvf Visium_FFPE_Human_Normal_Prostate_raw_feature_bc_matrix.tar.gz
wget https://cf.10xgenomics.com/samples/spatial-exp/1.3.0/Visium_FFPE_Human_Normal_Prostate/Visium_FFPE_Human_Normal_Prostate_spatial.tar.gz
tar -xzvf Visium_FFPE_Human_Normal_Prostate_spatial.tar.gz
This will create two subdirectories within the Visium Directory, titled “raw_feature_bc_matrix” and “spatial”. These subdirectories will contain barcode and expression information, or images and scaling information, respectively. Now, the Visium Directory may be inputted to createGiottoVisiumObject()!
A giotto object using either the hires or lowres image will be loaded depending on whether “tissue_hires_image.png” or “tissue_lowres_image.png” is provided to the png_name argument. In this example, the hires image will be plotted.
library(Giotto)
library(GiottoData)
VisiumDir = '/path/to/visium/directory/'
results_directory = paste0(getwd(),'/gobject_imaging_results/')
# Optional: Specify a path to a Python executable within a conda or miniconda
# environment. If set to NULL (default), the Python executable within the previously
# installed Giotto environment will be used.
my_python_path = NULL # alternatively, "/local/python/path/python" if desired.
# Optional: Set Giotto instructions
instrs = createGiottoInstructions(save_plot = TRUE,
show_plot = TRUE,
save_dir = results_directory,
python_path = my_python_path)
# Create a Giotto Object using Visium Data
FFPE_prostate <- createGiottoVisiumObject(expr_data = 'raw',
visium_dir = VisiumDir,
png_name = "tissue_hires_image.png",
instructions = instrs)
# visualize spots that are in tissue
spatPlot2D(FFPE_prostate,
show_image = TRUE,
cell_color = 'in_tissue',
save_param = list(save_name = 'high_res_IT'))
4 Standard Workflow#
4.1 Step 1: Invert Y-Values#
Before beginning, it is important to acknowledge that differences may exist in the conventions for defining coordinates within images and plots. As a result, it is often required to make the spatial location y values negative. This inversion is necessary for the spatial locations to appear in the same orientation as the image. This transformation of the spatial locations is automatically done for Visium datasets during createGiottoVisiumObject(). In the standard workflow, it is important to determine if this is necessary for the data at hand.
Why this inversion is necessary

Here, my_gobject refers to the giottoObject and my_spatlocs refers to the name of the spatial locations to which the image will be aligned.
# Retreive original spatial location data
spatlocs <- get_spatial_locations(gobject = my_gobject,
spat_loc_name = my_spatlocs)
spatlocs$sdimy <- -spatlocs$sdimy # Note the negative sign operator for inversion
# Overwrite the original spatial locations with the inverted ones
my_gobject <- set_spatial_locations(gobject = my_gobject,
spat_loc_name = my_spatlocs,
spatlocs = spatlocs)

4.2 Step 2: Create giottoImage#
For Visium datasets, scaling information is available in the scalefactors_json.json file found within the spatial subdirectory.
scalefactors_json.json for this dataset:
{"tissue_hires_scalef": 0.072809346, "tissue_lowres_scalef": 0.021842804, "fiducial_diameter_fullres": 304.63145798068047, "spot_diameter_fullres": 188.58137874994503}
Providing the appropriate factor to the scale_factor parameter will result in automatic alignment.
lowResPath <- paste0(VisiumDir,"Spatial/tissue_lowres_image.png")
lowResG_img <- createGiottoImage(gobject = FFPE_prostate,
mg_object = lowResPath,
name = "low_res",
scale_factor = 0.021842804)
Alignment values:
lowResG_img
## R TERMINAL OUTPUT:
# An object of class ' giottoImage ' with name low_res
#
# Min and max values are:
# Max on x-axis: 23520
# Min on x-axis: 5066
# Max on y-axis: -3682
# Min on y-axis: -23148
#
# Boundary adjustment are:
# Max adjustment on x-axis: 3949.001
# Min adjustment on x-axis: 5066
# Max adjustment on y-axis: 3682
# Min adjustment on y-axis: 2077.699
#
# Boundaries are:
# Image x-axis max boundary: 27469
# Image x-axis min boundary: 0
# Image y-axis max boundary: 4.547474e-13
# Image y-axis min boundary: -25225.7
#
# Scale factor:
# x y
# 0.0218428 0.0218428
#
# Resolution:
# x y
# 45.78167 45.78167
#
# File Path:
# [1] "/path/to/visium/directory/Spatial/tissue_lowres_image.png"
Without spatial locations
lowResG_img_no_locs <- createGiottoImage(mg_object = lowResPath,
name = "low_res_no_locs",
scale_factor = 0.021842804)
Alignment values:
lowResG_img_no_locs
## R TERMINAL OUTPUT:
# An object of class ' giottoImage ' with name low_res_no_locs
#
# Min and max values are:
# Max on x-axis: 10
# Min on x-axis: 0
# Max on y-axis: 10
# Min on y-axis: 0
#
# Boundary adjustment are:
# Max adjustment on x-axis: 0
# Min adjustment on x-axis: 0
# Max adjustment on y-axis: 0
# Min adjustment on y-axis: 0
#
# Boundaries are:
# Image x-axis max boundary: 10
# Image x-axis min boundary: 0
# Image y-axis max boundary: 10
# Image y-axis min boundary: 0
#
# Scale factor:
# x y
# 0.0218428 0.0218428
#
# Resolution:
# x y
# 45.78167 45.78167
#
# File Path:
# [1] "/path/to/visium/directory/Spatial/tissue_lowres_image.png"
Note that only default values are given to minmax and boundaries in this case.
With spatial locations, but also with do_manual_adj = TRUE
lowResG_img_manual <- createGiottoImage(gobject = FFPE_prostate,
mg_object = lowResPath,
name = "low_res_manual",
do_manual_adj = TRUE,
xmin_adj = 0,
xmax_adj = 0,
ymin_adj = 0,
ymax_adj = 0,
scale_factor = 0.021842804)
Alignment values:
lowResG_img_manual
## R TERMINAL OUTPUT:
#
# An object of class ' giottoImage ' with name low_res_manual
#
# Min and max values are:
# Max on x-axis: 23520
# Min on x-axis: 5066
# Max on y-axis: -3682
# Min on y-axis: -23148
#
# Boundary adjustment are:
# Max adjustment on x-axis: 0
# Min adjustment on x-axis: 0
# Max adjustment on y-axis: 0
# Min adjustment on y-axis: 0
#
# Boundaries are:
# Image x-axis max boundary: 23520
# Image x-axis min boundary: 5066
# Image y-axis max boundary: -3682
# Image y-axis min boundary: -23148
#
# Scale factor:
# x y
# 0.0218428 0.0218428
#
# Resolution:
# x y
# 45.78167 45.78167
#
# File Path:
# [1] "/path/to/visium/directory/Spatial/tissue_lowres_image.png"
4.3 Step 3: Add giottoImage to giottoObject and Visualize#
addGiottoImage() adds a list of images to the giottoObject specified. The name that the image is referred to as within the images slot is inherited from the name argument during createGiottoImage(). The default name is “image.”
# Since lowResG_img_no_locs is not associated with the gobject FFPE_prostate, it
# may not be added to the gobject.
FFPE_prostate = addGiottoImage(gobject = FFPE_prostate,
images = list(lowResG_img))
spatPlot2D(gobject = FFPE_prostate,
show_image = TRUE,
image_name = "low_res",
cell_color = "in_tissue",
save_param = list(save_name = 'low_res_IT'))

5 Manual Adjustment#
5.1 During giottoImage creation#
# createGiottoImage with manually defined adjustment values
lowResG_img_update_manual <- createGiottoImage(gobject = FFPE_prostate,
mg_object = lowResPath,
name = "low_res_update_manual",
do_manual_adj = TRUE,
xmin_adj = 5066,
xmax_adj = 3949,
ymin_adj = 2078,
ymax_adj = 3682,
scale_factor = 0.021842804)
FFPE_prostate = addGiottoImage(gobject = FFPE_prostate,
images = list(lowResG_img_update_manual))
spatPlot2D(gobject = FFPE_prostate,
show_image = TRUE,
image_name = "low_res_update_manual",
cell_color = "in_tissue",
save_param = list(save_name = 'low_res_update_manual_IT'))

5.2 After giottoImage creation, within the giottoObject#
# createGiottoImage with manually defined adjustment values
lowResG_img_to_update <- createGiottoImage(gobject = FFPE_prostate,
mg_object = lowResPath,
name = "low_res_to_update",
do_manual_adj = TRUE,
xmin_adj = 0,
xmax_adj = 0,
ymin_adj = 0,
ymax_adj = 0,
scale_factor = 0.021842804)
FFPE_prostate = addGiottoImage(gobject = FFPE_prostate,
images = list(lowResG_img_to_update))
spatPlot2D(gobject = FFPE_prostate,
show_image = TRUE,
image_name = "low_res_to_update",
cell_color = "in_tissue",
save_param = list(save_name = 'low_res_before_update_IT'))

# Use updateGiottoImage() to update the image adjustment values
FFPE_prostate = updateGiottoImage(gobject = FFPE_prostate,
image_name = "low_res_to_update",
xmin_adj = 5066,
xmax_adj = 3949,
ymin_adj = 2078,
ymax_adj = 3682)
spatPlot2D(gobject = FFPE_prostate,
show_image = TRUE,
image_name = "low_res_to_update",
cell_color = "in_tissue",
save_param = list(save_name = 'low_res_after_update_IT'))
