External Events
An External Event is temporal data used to inform a mission planner's decision making process. By nature, these events are external to Aerie itself and represent temporal occurrences that are immutable in nature.
Examples of External Events include:
- Ground station contacts
- Orbital events
- Weather forecasts
In this document, we will discuss what External Events are, what their containing entities (External Sources) are, their grouping mechanisms (Derivation Groups), as well as how they manifest within Aerie.
Concepts
Data Constructs
With External Events (and External Sources) come several constructs.
External Events
External Events represent temporal occurrences outside of the locus of control of the modelled spacecraft(s). See External Events below for more information.
External Sources
External Sources are containers for External Events and the primary way that users interact with External Events. To get External Events into Aerie, an External Source must be uploaded that contains the event definitions inside. See External Sources below for more information.
External Event Types
External Event Types define the typing of each External Event. This typing groups External Events on the basis of the data they represent, and details the allowed attributes that a given External Event is allowed to have. For example, an External Event may be of the type Ground Station Contact, Solar Flare, or Weather Forecast. Each of these types contains a schema for a set of allowed/defined attributes which are then implemented by the events using this External Event Type to display additional data, such as station, flux level, or precipitation.
External Source Types
External Source Types define the typing of each External Source. This typing groups External Sources on the basis of the data they contain, and details the allowed attributes that a given External Source can have to describe the data contained within it. For example, you may have an External Source Type of Weather Report to represent all your weather reports, and inside each of the Weather Report-typed sources you can expect to find External Events that share a common External Event Type that appears in all Weather Report-typed sources (ex: Temperature). Identical to External Event Types, External Source Types also contain attribute schemas that are then implemented by the External Sources using the External Source Type.
Derivation Groups
Derivation Groups are collections of External Sources that can be associated with plans. Within a Derivation Group, the collective External Events from its member External Sources are used to create a derived set of events for the Derivation Group. See Derivation below for more information.
Derivation Group & Plan Associations
Derivation Groups can be linked to a plan (or plans) which allows their derived events to be utilized within the plan for visualization, planning, and scheduling. See UI Usage below for more information on use within the UI.
External Events
An External Event is a construct that represents the occurrence of something outside of what is modeled in the mission model. They could be events that are close to the spacecraft and directly affect behavior on it, like a Ground Contact event that might affect when communications happen, or they could be much more remote, like a constellation event. The main concept with External Events is that they are immutable events that occur at some point in time (either instantaneously, or for some set duration) and can/will impact mission planning. Functionally, these are at the interstice between external datasets and activities, in that they manifest on the timeline as activities (discrete blocks of time), but they themselves do not affect the mission model or invoke any actions similar to external datasets.
Why?
The need for these became apparent after an exploration of External Events implemented as activities and as resources. To mimic the same construct as a resource, we would need to parse the external data and then place it on the timeline as a discrete resource, which given the fact that there are gaps between events and there can also be simultaneous events proved extremely unwieldy and awkward. While activities were a more natural representation, we still required an intermediary of a resource to schedule these activities based off of an ingested file, as it isn't possible to schedule activities exclusively following information in a file.
Currently, External Events are not accessible to the procedural constraints engine. However, this is in the planned scope for External Events, and future releases will add this capability
External Sources
External Sources represent a collection of External Events. When we upload External Events, we can only do so by uploading them in an External Source.
These sources are defined using JSON Schema format with a meta-schema enforced to ensure certain fields such as key and start_time are present and in the correct format.
Schema
There are two parts to each External Source: 1) a source header and 2) a collection of events:
{
"source": {
// fields
},
"events": [
// events
]
}
Source
The source field includes the following fields:
key(required)- Unique identifier for the External Source
- Uniqueness must hold across the Derivation Group that it is a a part of
- Typically a filename or combination of details about the source itself (i.e., an external version number, spacecraft identifier, etc.)
- Example:
ExampleExternalSource:example-external-source.json
source_type_name(required)- Used to classify the External Source, categorize for grouping/filtering within the UI & Derivation Groups (see External Source Types for more information), and applying the schema defined for the type to the attributes of the External Source being uploaded (see Attributes for more information)
- Example:
ExampleSourceType
derivation_group_name(optional)- Defines the Derivation Group that this External Source should become a part of once uploaded. See Derivation for more information.
- If
derivation_group_nameis omitted in the source file, it will need to be specified on upload. - Example:
ExampleSourceType Default
valid_at(required)- Similar to a version number, though it describes at exactly what real-world time a file becomes valid
- Can be sorted and ordered like a version number, but additionally provides extreme detail about when it can/should be utilized which plays a key role in derivation
- Example:
2022-001T00:00:00Z
period(required)- Specifies the exact range of simulation/plan time over which this external source applies.
- Multiple External Sources of the same External Source Type may cover the same
period, which is wherevalid_atcomes into play - see derivation for more information - Composed of a combination of
start_timeand eitherend_timeorduration
- Multiple External Sources of the same External Source Type may cover the same
- Example
...,
"period": {
"start_time": "2022-001T00:00:00Z",
"end_time": "2022-008T00:00:00Z"
},
...
- Specifies the exact range of simulation/plan time over which this external source applies.
attributes(optional)- An object that includes additional data relevant to the External Source in question.
- In order for an External Source to have attributes, its source type must define a schema for those attributes.
- For a Weather Forecast file, this may include details about the location, which version this file is, and notes about what changed between this version and the last.
- The
attributeskey can be omitted for sources whose type has no attributes. - Example
...,
"attributes": {
"location": "Greenbelt, MD",
"version": 2,
"changelog": "Updated precipitation likelihood for 2024-007T00:00:00 from 20% to 35%."
},
...
Events
Events are represented as a set of objects contained in a list, and each event has the following fields:
key(required)- Unique identifier for the External Event
- Uniqueness is required within the External Source, but not across different External Sources. For more information, see derivation
- Example:
ExampleEvent:1/sc/sc1:1:X/1
- Unique identifier for the External Event
event_type_name(required)- Used to classify the External Event, and categorize for grouping/filtering within the UI & Derivation Groups (see External Event Types for more information)
- Example:
GroundContact
start_time(required)- Specifies the start time of the External Event within simulation/plan time
- Example:
2022-001T00:00:00Z
duration(required)- Specifies how long the External Event lasts in simulation/plan time (represented in
HH:MM:SS) - This value may be
00:00:00to represent an instantaneous event (ex: time of sunrise/sunset) - Example:
00:30:00
- Specifies how long the External Event lasts in simulation/plan time (represented in
attributes(optional)- Similar to attributes for External Sources, this property lists various characteristics of a given event not directly implied by the above fields' values.
- Must follow the schema as defined by the External Event Type
- If the External Event Type does not define an attributes schema, this key may be omitted.
- For a Weather Forecast file, a given event may require properties detailing the likelihood of precipitation, or the wind speed.
- Example:
...,
"attributes": {
"precipitation_lik": 0.35,
"wind_speed (mph)": 3
},
...
Full External Source & Event Types Example
The following is a full example of a valid, JSON representation of an External Source Type and External Event Type. The type definitions must be uploaded to Aerie prior to attempting to use them in an External Source and/or External Event(s).
The example represents the definitions for a weather forecast source & event.
{
"event_types": {
"WeatherReport": {
"type": "object",
"required": ["daily_high", "daily_low", "precipitation_lik"],
"properties": {
"daily_high": {
"type": "object",
"properties": {
"value": {
"type": "number"
},
"units": {
"type": "string"
}
},
"required": ["units", "value"]
},
"daily_low": {
"type": "object",
"properties": {
"value": {
"type": "number"
},
"units": {
"type": "string"
}
},
"required": ["units", "value"]
},
"precipitation_lik": {
"type": "number"
}
}
}
},
"source_types": {
"Weather": {
"type": "object",
"required": [
"location",
"version",
"changelog"
],
"properties": {
"location": {
"type": "string"
},
"version": {
"type": "number"
},
"changelog": {
"type": "string"
}
}
}
}
}
For more information on External Source & Type definitions & attribute schemas, see External Events and Source Attributes
Full External Source Example
The following is a full example of a valid, JSON representation of an External Source that can be ingested by Aerie.
The example represents a week-long weather forecast where each event is the report for a specific day.
{
"source": {
"key": "WEATHER_2024001_2024008_IDEAL.V00.json",
"source_type_name": "Weather",
"derivation_group_name": "Weather Default",
"valid_at": "2024-001T00:00:00Z",
"period": {
"start_time": "2024-001T00:00:00Z",
"end_time": "2024-008T00:00:00Z"
},
"attributes": {
"location": "Greenbelt, MD",
"version": 2,
"changelog": "Updated precipitation_lik for 2024-007T00:00:00 from 20% to 35%."
}
},
"events": [
{
"key": "W/Day/001",
"event_type_name": "WeatherReport",
"start_time": "2024-001T00:00:00Z",
"duration": "24:00:00",
"attributes": {
"daily_high": {
"value": 43.2,
"units": "F"
},
"daily_low": {
"value": 32.9,
"units": "F"
},
"precipitation_lik": 0.7
}
},
{
"key": "W/Day/002",
"event_type_name": "WeatherReport",
"start_time": "2024-002T00:00:00Z",
"duration": "24:00:00",
"attributes": {
"daily_high": {
"value": 40.5,
"units": "F"
},
"daily_low": {
"value": 29.2,
"units": "F"
},
"precipitation_lik": 0.6
}
},
{
"key": "W/Day/003",
"event_type_name": "WeatherReport",
"start_time": "2024-003T00:00:00Z",
"duration": "24:00:00",
"attributes": {
"daily_high": {
"value": 42.1,
"units": "F"
},
"daily_low": {
"value": 28.4,
"units": "F"
},
"precipitation_lik": 0.5
}
},
{
"key": "W/Day/004",
"event_type_name": "WeatherReport",
"start_time": "2024-004T00:00:00Z",
"duration": "24:00:00",
"attributes": {
"daily_high": {
"value": 50.0,
"units": "F"
},
"daily_low": {
"value": 32.1,
"units": "F"
},
"precipitation_lik": 0.4
}
},
{
"key": "W/Day/005",
"event_type_name": "WeatherReport",
"start_time": "2024-005T00:00:00Z",
"duration": "24:00:00",
"attributes": {
"daily_high": {
"value": 46.3,
"units": "F"
},
"daily_low": {
"value": 21.0,
"units": "F"
},
"precipitation_lik": 0.3
}
},
{
"key": "W/Day/006",
"event_type_name": "WeatherReport",
"start_time": "2024-006T00:00:00Z",
"duration": "24:00:00",
"attributes": {
"daily_high": {
"value": 34.3,
"units": "F"
},
"daily_low": {
"value": 22.5,
"units": "F"
},
"precipitation_lik": 0.2
}
},
{
"key": "W/Day/007",
"event_type_name": "WeatherReport",
"start_time": "2024-007T00:00:00Z",
"duration": "24:00:00",
"attributes": {
"daily_high": {
"value": 39.0,
"units": "F"
},
"daily_low": {
"value": 35.3,
"units": "F"
},
"precipitation_lik": 0.35
}
}
]
}
External Sources expect a start_time and an end_time, whereas External Events expect a start_time and a duration.
Derivation
Derivation was an operation conceived for the reconciliation of overlapping External Sources. The problem at hand was that it is entirely possible for sources to be generated at later points in time, that cover slightly different windows of plan time than their predecessors.
If this is the case, some reconciliation scheme is needed to determine which External Events from these External Sources are valid and should be visible on the timeline.
Reconciliation Mechanism
To begin this discussion, we should first talk about when External Sources apply.
External Sources, as discussed earlier, have within them a valid_at field, which was described as a more particular form of a version number. With this, we know we can impose an ordering of External Sources. Given that ordering, figuring out when each External Source actually applies is as simple as determining, for a given time (or slot of time), what the most recently valid External Source is. For example, the following set of 4 External Sources would be applicable in the following order. Note that one of the External Sources, B, is valid twice, as it is the most recently valid External Source for a given slot of time:
Given this, we can now discuss how to derive External Events from a Derivation Group of External Sources. There are 4 rules (though the first two are effectively cases of each other):
An External Event superseded by nothing will be present in the final, derived result.
The logic behind this is that this External Event has no future data that might challenge its validity, so it reasonably would trickle down as an event in a final result.
An External Event partially superseded by a later External Source, but whose start time occurs before the start of said External Source(s), will be present in the final, derived result.
The logic behind this is similar. This External Event, at its start, has no future data that might challenge its validity. Because its start is still considered valid, we accept the event in its entirety to the final, derived result, as no future External Source that could overlap with this External Event could place an External Event that applies to the time that this External Event starts at. If it did, we have a case of rule 3.
An External Event whose start is superseded by another External Source, even if its end occurs after the end of said External Source, will be replaced by the contents of that External Source (whether they are blank spaces, or other events).
The logic here is simply that we now have more recent data, so we can safely do a complete replacement. Should an External Event be replaced despite ending after the original source, we can still justify this as the updated External Source - if it had need for such an External Event - would have included it.
An External Event that shares an ID with an External Event in a later External Source will always be replaced.
By introducing naming, we can allow replacement.
Derivation Example
A diagram illustrating each of the derivation cases follows:
The following explains what is happening to each External Event in the example:
External Source A
2:D
- Status: Excluded
- Rules
- 3: Even though its end is after end of External Source B, the start is subsumed. In any case, it is wholly subsumed by External Source C.
- 4: Replaced by key in External Source C (regardless of the fact that the new event has a different type).
7:C
- Status: Excluded
- Rules
- 3: Even though its end is after end of External Source B, the start is subsumed.
8:B
- Status: Included
- Rules
- 1: This External Event is subsumed by nothing.
External Source B
1:A
- Status: Included
- Rules
- 1: This External Event is subsumed by nothing.
2:A
- Status: Excluded
- Rules
- 3: Completely subsumed by External Source D.
- 4: replaced by key in External Source C (regardless of the fact that the new event has a different type).
3:B
- Status: Included
- Rules
- 2: Starts before any other External Source, despite end being subsumed.
4:B
- Status: Excluded
- Rules
- 3: Completely subsumed by later External Source.
External Source C
5:C
- Status: Included
- Rules
- 1: This External Event is subsumed by nothing.
6:C
- Status: Included
- Rules
- 1: This External Event is subsumed by nothing.
2:B
- Status: Included
- Rules
- 1: This External Event is subsumed by nothing.
- 4: Also most recent occurrence by key name, so this is the only one that persists
External Source D
- 9:C
- Status: Included
- Rules
- 1: This External Event is subsumed by nothing.
- 9:C
Derivation Groups
The final question remaining is: what set of external sources will this derivation operation be run against? One possible candidate for this could be the External Source Type. External Events that share an External Source Type include External Events of the same types, and will likely overlap in key namespace as well. This is a nearly ideal solution except in the case of contingencies.
Imagine we have 4 sources of the same source type. Two of these relate to an ideal contingency, and two of these relate to a degenerate contingency:
1) Ideal
DSN_Contact_Ideal_2026001007.jsonDSN_Contact_Ideal_2026003010.json
2) Degenerate
DSN_Contact_Degen_2026001007.jsonDSN_Contact_Degen_202603010.json
If we were to derive solely based on External Source Type, we would effectively be stacking all of these sources against each other to produce one result:
Given that these are different cases (despite sharing a source type) it would be useful to have different groups under which derivation is possible. This way, we don't conflate sources just for sharing a source type:
As such, we now define the notion of a Derivation Group. A Derivation Group defines a subgroup within an External Source Type, of External Sources to derive against each other. Derivation Groups are what we associate with plans (as opposed to individual, un-derived External Sources, or entire External Source Types).
UI Usage
External Events show up in the UI in two main places - an External Source Manager, and in plans (both in panels and timelines). Our discussion of UI features will therefore be divided along which page the features appear on.
There is also a tutorial section that walks through the content of this section in a step-by-step fashion, albeit with less detail.
External Source Manager
The External Source Manager page handles the uploading & inspection of External Sources, as well as the creation and management of Derivation Groups.
Defining External Source Types and External Event Types
In order to use attributes in External Sources or Events, the corresponding Source and Event Type schemas must first be defined in a schema file. You can find more about the structure of schema files here.
To upload a schema file, navigate to the Types tab in the External Source Manager.
Use the left pane to upload a new schema file.
This page displays all of the types that have been defined and the attributes they have. Here, we see the Weather source type has three attributes: version (a number), location (a string), and changelog (a string).
Uploading an External Source
The main view allows users to upload External Sources. Most of these fields autofill and are immutable once parsed - except the Derivation Group which will be autofilled in the style of external_source_type Default, but can be edited by the user before upload.
Inspecting an External Source
After uploading an External Source, it will show up in the table on the upper-right pane. This table can be filtered by External Source Type as well as sorted by each of the columns - note the default multi-sorting on the columns Source Type, Derivation Group, and Valid At, which is meant to present the sources in the easiest-to-read manner (grouping by their common types & Derivation Groups, then ordering by their Valid At times).
Selecting a source allows for the inspection of the source, which is essentially an in-Aerie rendering of what's included under source in the uploaded JSON (as discussed earlier). It is here, as well as in the table, that deletion can be performed. Note that for deletion to work, an External Source must not be associated with any existing plan (that is the Derivation Group it is a part of must be dissociated from all plans before deletions become legal). Aside from that, there is currently no system of ownership, meaning admins and users alike, as long as External Sources satisfy the aforementioned condition, can delete at will. Viewers, however, cannot.
It should also be noted that all this inspection takes place outside the context of a plan, so that External Sources & their events can be viewed without having to open and associate the External Source(s) to a plan.
Finally, it is also possible to inspect each source's contained events as a table. This is useful, as Derivation Groups on plan timelines only show what has been derived, meaning anything that has been selected against won't appear. This page is therefore the absolute truth to see everything that is included, derived-out or not, with a given External Source.
Plan
Once an External Source has been uploaded, it can be viewed and interacted with in the context of a plan.
To begin working with external sources in the plan, the left side panel should be set to the "External Sources" option. This panel is where users (administrators or plan collaborators) can associate different Derivation Groups with the plan.
Derivation Group Management
There are currently two levels to linking a Derivation Group to a plan: association and enabling. Association is handled here, via the Manage Derivation Groups button. This presents a modal where users can expand different Derivation Groups to view the associated sources and select whether or not they would like to associate said Derivation Group with their plan.
Association simply means that the plan is aware of the existence of the Derivation Group, and is the gatekeeper for whether it is visible in the External Sources panel. In this panel lies the second level to linking, which is enabling. Enabling simply dictates whether a Derivation Group (even if the filters in the Timeline Editor select for it) is visible on the timeline. Disabling hides all events associated with the group completely, whereas enabling does the opposite. It does not, however, dissociate the plan and Derivation Group. This is simply a visibility attribute and is actually persisted as part of the view (and even persists after Derivation Group dissociation/reassociation, meaning if a Derivation Group is dissociated and then re-associated, its "enabled" setting stays the same).
Once a Derivation Group has been associated with the plan, new external event sources may be added to it (or deleted from it), causing the derivation process to automatically re-run, and affecting the events which are shown on the plan timeline. For this reason, this panel also provides users with a notification of recently added and deleted sources which affect this plan, and their respective External Source Type and Derivation Group. This notification persists until acknowledged with the Dismiss button.
| Enabled | Disabled |
|---|---|
 |  |
Timeline
The timeline shows any events from Derivation Groups that are associated with the plan, enabled in the current view, and are filtered for in the timeline editor. Having covered the first two, we can now discuss the editor and its effects on the timeline.
The layer of the timeline is modelled after the activity layer, as it provides an easily readable representation of discrete-time occurrences. Layers share the layer options also provided to activity layers, as well as implementing a 'Group By' option specifically for external events. The 'Group By' mechanic allows the user to have external events either grouped by their external source (within the derivation group), or their event types.
| 'Group By' External Source | 'Group By' External Event Type |
|---|---|
 |  |
Tables and Inspection
The screenshot below shows a brief overview of the remaining panels that have been added to plans. It is possible to either mouse over External Events for additional detail or to select them in the timeline, which selects them in a table view beneath the timeline as well as details them in the right pane. This looks as follows:
A Note on Views
As a final note on the frontend, it is worth briefly detailing what parts of the plan page additionally get persisted when a view is saved:
- Derivation Group enablement - whether a Derivation Group is enabled or not in the External Sources panel,
- External Event Type filters - the External Event Type filters selected in the timeline editor,
- External Event options - the options in the timeline editor filter.
Scheduling Activities from Events
External events associated with a plan are accessible from the Procedural Scheduling API, just like resources and activities. This allows users to write scheduling goals which create activities based on the presence (or absence) of external events - for example, "create a downlink activity 5 minutes after the start of every 'DSN Contact' type event".
To access events in a procedural goal, you must create an External Events timeline object - see Timelines: External Events. A full example of a scheduling goal using events can be found on the Procedural Scheduling Examples page.
Currently, external events are not accessible from the Declarative Scheduling EDSL - but we may implement support for this in the future.
What Remains
Here, we discuss briefly everything that is not currently implemented but that we do plan to in the future.
Ownership/Roles
Our current solution to a lack of clear ownership/roles around External Sources is as follows:
- any user and any administrator can add External Sources,
- as there is no ownership yet, any user and any administrator can delete External Sources as long as the Derivation Group they are a part of is not associated with any plans, and
- associations between plans and Derivation Groups can be handled by any administrator or a user that is collaborating on that plan.
It might make sense to add ownership, i.e. a given user owns an External Source and therefore has privileges to delete it (much like how plans are implemented, except External Sources cannot be edited), but as it would primarily be used for deletion functionality but would introduce a lot of complexity for this simple problem, it was sidelined entirely.
That being said, as we move forward with more features relating to External Events, more nuance may come to the problem of ownership and roles. Once plans start actually depending on these plans, the rules surrounding ownership and permissions may need to change or be entirely rethought. Ultimately, we need something that justifies the implementation of such a scheme in that it provides more than the status quo.