[GSoC’19|VisMa|Mayank] Week 1 to 3

This blog post is intended to pen down my experience of working with aerospaceresearch.net and on VisMa – Visual Math as a part of Google Summer of Code 2019. A part of this blog post will be about how it feels to be working here & other parts will focus on technicalities of the project (about how VisMa has been improved over the past three weeks).

Firstly, working with aerospaceresearch.net has been a great experience. The mentors of the project Shantanu, Manfred & Siddharth are very helpful. We almost regularly have chat on Zulip (& sometimes on WhatsApp) wherein we discuss the plans for the project and about implementation. Owing to these healthy discussions the project development is going at a fast rate (which is indeed a good thing!). The code base of VisMa is very well written and the complex task of simplifying mathematical equations has been beautifully coded. About the workload, I work regularly for 5 – 6 hours, but sometimes when a crucial bug finds its way in, the work time has to be increased. Overall it’s a Fun & Learn experience.

Now let’s talk about what we (the VisMa Team) have achieved so far. During the community bonding period, I spend my time to get equipped with basics of PyQt (on which the GUI of VisMa is based) and spend the remaining time studying the tokenizing and simplify modules. Honestly, these modules were hard to understand. But the effort turned fruitful during the coding period. Also, I completed the task of simultaneous equation solver during this time. The logic was easy to frame, but the inclusion of comments and animations was tricky and involved the entire redesigning of the LaTex generator module (the part which created the final latex to be displayed in GUI).

Over to the first week of the coding period, the task which was decided was to re-design all the simplify module in an object-oriented fashion, which before coding period I believed would be most difficult to write (after all more than 2000 lines of complex Pythonic logic, to be honest, was scary). But, efforts to understand module during the Community Bonding Period turned out to be fruitful, and I was happy to achieve this task in the desired time. Like any other programming project, there are some minor bugs out there, which will be fixed in the upcoming time!

Coming to week second, we decided to work on Expression Simplification. Now, what expression is, briefly (& vaguely) any part of inputted equation enclosed in brackets can be termed as an Expression. So like when we solve an equation by hand, we solve the bracket part first, our task was to make visma do same! Earlier it didn’t support Expression simplification. For that, we decided to go for a complex recursive logic (solving the inner expression first then the outer expressions & so on). Recursive logic was complex & time consuming to implement, taking care of all comments & animations during recursion was a challenge. It took somewhat 9 days to come up with something fruitful. But yes it was worth it, VisMa could easily solve Expressions now. A happy moment indeed! 🙂

Next phase was to include a new module to VisMa. It was decided it has to be a Discrete Math module (‚cause why the heck not!). We decided to go for Combinatorics stuff first, thus implementing Combinations & Permutation stuff. These modules require factorial of a number, I got this idea why not to implement factorial as a separate module and then use in Permutations & Combinations stuff. This added another factorial feature to VisMa.

For now, I am working of adding triple degree equation solvers to project. We as of now have decided to do so using Cardano Algorithm. If this turns out well it will also be done for four-degree equations. After this will be done, VisMa will be capable of solving & showing detailed steps for up to 4-degree equations.

Below is a short GIF showing some of new features in action:

Hope you enjoyed reading Chapter One of my Developer Diary. From now on, I will write these blogs bi-weekly.

Chears to Code, Space & Maths!

[GSoC2019|PlaneSpotting|Shoumik] Introduction: Self – Synchronisation

Hi there!

My name is Shoumik Dey. I am currently a second year student at Manipal Institute of Technology, India. I preserve strong interest in aviation, aeromodelling and maybe this is the reason that I have chosen to work on this project, this summer, for Google Summer of Code 2019.

You can also find me here:

This is the first blog post for this project. This post will describe the work done so far, the current outcomes, hurdles faced and also they were/can be be solved.

Introduction

‚ADS-B‘ stands for Automatic Dependent Surveillance–Broadcast. The aircraft automatically brodcasts each frame on the 1090MHz frequency periodically which contains navigational data, such as, location, altitude, airspeed etc.

All ADS-B frames do not contain location information in them, therefore the ultimate goal of this project can be described in two parts:

  • Self-syncronisation of the ground stations in focus without an external agent.
  • Determining the location of the aircraft when location data is not received using multi-lateral positioning

Self-synchronisation

The current and widely used method of synchronising the clock of any ground based station is by using GNSS(Global Navigation Satellite System)-receiver interrogation. The local clocks gets aligned with the atomic clocks in the satellite.

But in this case, the synchronisation takes place by calculating the actual point of time when an aircraft broadcasts a message. This serves as the reference time at all ground stations, and since this time value is same all over, hence all the stations become self-synchronised.

Data required for self synchronisation:

  • Time of arrival of the message at the ground station
  • Time difference of arrival at that station.

Calculating the time difference of arrival(TDOA):

  1. The receiver provides the time at which the frame(with location) is received.
  2. The time of travel of the frame is calculated by using simple math and that is subtracted from the time of arrival.
sqrt((x2-x1)^2 + (y2 - y1)^2 + (z2 - z1)^2)

where x1, y1, z1: Latitude, longitude and altitude of ground station; x2, y2, z2: Latitude, longitude and altitude reported by aircraft.

Data that we have for self-synchronisation:

We have decided on using dump1090 for generating the ADS-B messages from the IQ stream as recorded by the SDR radio. Dump1090 provides the mlat(multilateration) data in avr format. The first 6 bytes of the message provides the sample position of the last bit of that message.

Example:

Raw: @0000000929E28e3ff6e6990c4684000011548194;
Sample Position: 0000000929E2
Message: 8e3ff6e6990c4684000011548194

Next, we shift the sample position record from the last bit of the message to the first bit.

Typical recording of a 112 bit ADS-B message
Typical recording of a 56 bit ADS-B message

As we can see, the length of the messages is exactly half of each other.

The sample position reported by dump1090 is exactly 240 samples ahead of the preamble (You can see the preamble at the beginning, by 4 spikes in the signal).

So, the sample position is taken back by 240 samples for both the cases to reach the beginning of the preamble.

Now, data in hand:

  • Start time of the stream
  • Sample position of the beginning of the message in the stream.

Therefore the time of arrival(TOA) can thus be calcualted.

Work done so far

  • A plugin to import the mlat output of dump1090 (ADS-B raw) has been created, which imports the message and the sample position into the main program
  • The raw data is reorganised in our suitable format and terms.
  • With each frame present;( for 112 bit frames as of now)
    • Determination of the type of frame and it’s content
    • Decoding those data present in that frame
  • Output dump file which contains the frame and the data contained in it. (Only extraction, no calcualation)

Next up….

The next blog post will contain the implementation of the calculation layer of the project. In this layer, the data in the frames will be processed and information such as position, velocity, altitude will be covered.

The post will also contain the unusual findings and irregularities that were found in the stream and how we have decided to deal with it.

[GSoC2019|DirectDemod|Vladyslav M] BigImage and BigGlobe.

This blog post will cover the process of visualization of NOAA images, specifically we will talk about general visualization pipeline and different ways to visualize satellite imagery, including creating interactive web maps and virtual globes. 

Visualization pipeline

We shall start from the very beginning. NOAA satellites constantly scan the Earth’s surface and then transmit it as a radio wave signal. This signal is captured by SDR (Software-defined radio system) and saved in wav format. Next step is dedicated to decoding this signal, i. e. finding sync signals, extracting and decoding an image line-by-line.

The decoded image includes both sync signals and 2 variants of the scanned image (in different color schemes), therefore it requires one more preprocessing step, where it is being cropped and rotated. Next two images are the results of this step, they will be used as an example for visualization.

Figure 1. Example of decoded image.

Figure 2. Example of decoded image.

Next step is georeferencing, which has already been described in previous post [1]. On this step, the coordinate system will be assigned to the image, every pixel will have a corresponding geographic coordinates. Main computation is done with gdal library.

Merging images

The goal of the project is to primarily create tools, that could visualize a set of satellite images. It means that we must find a way to combine several images into one standalone raster, which we could then display as a web map or as a virtual globe. 

Using tools from image analysis (image stitching) would be a bad approach, because of two reasons:

  • these methods are approximate in their’s nature
  • do not take in account image distortions

A better approach would be to use pixel’s coordinates. We know the coordinates of each pixel, thus we could tell exactly where it will be located on the new image. For overlapping areas, for each pixel we will have one of 3 cases:

  • no pixels mapped to this pixel
  • 1 pixel is mapped is mapped to this pixel
  • several pixels mapped to this pixel

If no pixels have been map to the position, then it will be filled with special no-data value. If there is one pixel, then it’s value will be assigned to the corresponding pixel on new image. If there are several candidates for certain pixel, we will compute new pixel value from them based on some strategy, for example taking max or average. In this example we’ll use averaging for overlapping regions. Below is an example of merging 2 images above, country boundaries added for clarity.

Figure 3. Example of merged satellite images.

As you may see, there is still a contrast present between the images and the overlapping part, the reason for this are different lightning conditions for images. Nevertheless, image quality with this method outperforms other methods, like taking maximum for example.

Creating TMS

On this step our raster will be transformed to yet another representation, which will be used for further visualization.

TMS (Tile Map Service) – is a protocol for serving maps as tiles i. e. splitting the map up into a pyramid of images at multiple zoom levels [2]. See the image to better understand the concept.

Figure 4. Tiles pyramid.

So for each level zoom level a set of tiles is generated, each tile has it’s own resolution and will be displayed at specific position at specific zoom. To generate this set of tiles gdal2tiles.py program is used, it transforms an input raster into a directory tree with tiles [3].

It should be noted that TMS is not the only choice here, another alternative is WMS (Web Map Service) [4]. 

Interactive web-maps

Finally, having TMS tiles computed we could create a web map. To do this we will use leaflet.js, which is a library for web-mapping applications. [5] We will combine two layers, the first one is our tiled raster, which will be loaded from local directories and the second one is Open Street Map layer which will be loaded from OSM site. The code for map is contained in the map.html file and could be opened directly in browser. You can see a live example here https://caballeto.github.io/index.html, or in the iframe below.

Virtual globe

Another way to visualize satellite images is of course to plot them on virtual globe. Once again we could use already computed TMS tiles here. To render the globe we will use webglearth2 library, which is created exactly for this purpose. Similarly to a web map, we will add an OSM layer as a background layer. To open it locally you will have to start an http server, and then open the globe.html via http. Running commands (run from directory where globe.html is stored): 

  • python -m http.server 8000
  • go to browser and connect to localhost:8000/globe.html.

Or you can simply take a look at an online version below, better view at https://caballeto.github.io/globe.html.  

Software

To create presented visualizations for a raster, you can use generate_map.py tool from directdemod package. It will automatically create tiles from your raster and then will generate corresponding map.html and globe.html files.

Conclusion

In this post we’ve looked at different ways and tools for raster visualization, specifically we’ve analysed the visualization pipeline for satellite imagery. Presented types of visualizations are compatible with many different mapping applications (leaflet, openlayers etc.) and could be easily published and used on the Internet. 

References

  1. https://aerospaceresearch.net/?p=1248
  2. https://wiki.openstreetmap.org/wiki/TMS
  3. https://gdal.org/programs/gdal2tiles.html
  4. https://en.wikipedia.org/wiki/Web_Map_Service
  5. https://leafletjs.com

[GSoC2019|DirectDemod|Vladyslav M] Installing DirectDemod. Map overlays.

This post will cover the installation process of DirectDemod package, we’ll also talk on how to create pretty map overlays using the directdemod.georeferencer module.

Installing DirectDemod

Recently, the installation process for DirectDemod has been updated (currently on Vinay_dev and Vladyslav_dev dev branches only). You must have anaconda or miniconda distribution installed. From here do the following:

  1. Clone repository:
    • git clone https://github.com/aerospaceresearch/DirectDemod
  2. Change branch to Vinay_dev
    • git checkout -b Vinay_dev
  3. Create environment
    • conda env create -f environment.yml -n my_env
  4.  Activate the environment
    • conda activate my_env
  5. Add directdemod module to your PYTHONPATH.
  6. Test installation with pytest.

Create fancy map overlays

Before creating map overlays we need to have a georeferenced image, let’s make one. samples/decoded directory contains 2 already decoded and preprocessed NOAA images ready for georeferencing. Create an empty file – SDRSharp_20190521_170204Z_137500000Hz_IQ.wav in samples directory, it will serve as a placeholder; the image was already extracted but the name of SDR file contains valuable information needed for georeferencing. 

To georeference the image, we will first extract information from the name of SDR file. Go to directdemod directory and type the following command.

python misc.py -f ../samples/SDRSharp_20190521_170204Z_137500000Hz_IQ.wav -i ../samples/decoded/SDRSharp_20190521_170204Z_137500000Hz.png

misc.py extracts data about satellite’s orbit and position, and then embeds it in json format into created tif file.

 Now we will use georefencer.py CLI interface to georeference the image and create a map overlay.  -m or –map option tells the georeferencer to overlay the image with map boundaries after georeferencing.

python georeferencer.py -m -i ../samples/decoded/SDRSharp_20190521_152538Z_137500000Hz.tif

Finally, the georeferenced image.

Figure 1. Georeferenced image combined with country boundaries.

Note on shapefiles

A shapefile is a vector data storage format for storing geographical features [1]. In the context of this application, country boundaries are being represented as a shapefile. Inside the shapefile points may be stored as polygons or as polylines, overlaying an image with polylines will have an effect of overlaying country frontiers, while overlaying an image with polygons will have an effect of overlaying country areas. 

Figure 2. Georeferenced image combined with polygons shapefile.

Further work

The next blog post will discuss merging NOAA satellite images, including different techniques for handling overlapping areas cases.

References

  1. https://en.wikipedia.org/wiki/Shapefile