Visual understanding of the GAE data warehouse

Question

I’m trying to understand how the Google App Engine (GAE) data warehouse is created and how to use it. It's a little difficult for me to visualize the structure from the description on the top of the page page .

Can someone explain the data warehouse using numbers for us, people-oriented? Or, again, point out a good visual tutorial?

I am specifically looking for answers with charts / numbers that explain how GAE is used.

Answer 1

Very simplified. I realized that GAE can be thought of as a hash map of hashmaps.

This means that you can view it like this:

Answer 2

The IO 2008 session “Under the covers of the Google App Engine data warehouse” has a good visual overview of the data warehouse.

https://sites.google.com/site/io/under-the-covers-of-the-google-app-engine-datastore

http://snarfed.org/datastore_talk.html

For more IO negotiations go to: https://developers.google.com/appengine/docs/videoresources

Answer 3

I think there is no right answer, just different models of the mind. You may find mines, troublesome, or both, depending on your programming background. I present the data warehouse as a single huge distributed collection of value values, consisting of all entity data of any type in any namespace and all GAE applications for all users. One bucket is called a group of entities. It has a root key that (under the hood) consists of your appID, namespace, view, entity identifier or name. A group of persons contains one or more objects that have keys that extend the root key. An entity that belongs to the root key itself may or may not exist. Operations within one group of objects are atomic (transactional). An entity is a simple cartographic data structure. The 2 built-in indexes (ascending and descending) again represent 2 giant sorted sets of index entries. Each index record represents an appID, namespace, kind data structure, property name, property type, property value, entity key - in that order. Each (automatic) indexed value of each property of each object creates 2 such index elements. There is another index in which there are only entities. However, custom indexes move on to another sorted collection with elements containing appID, namespace, index type, combined index value, entity key. This is the only part of the entire data warehouse that uses metadata. It stores an index definition that tells the store how the combined index value is generated from the object. This is a picture that was burned in my mind and from which I know how to make a happy data warehouse.