Detailed documentation of all internal modules
Todo
This is not yet complete, see issue #52
app
- flexmeasures.app.create(env: Optional[str] = None, path_to_config: Optional[str] = None, plugins: Optional[List[str]] = None) flask.app.Flask
Create a Flask app and configure it.
Set the environment by setting FLASK_ENV as environment variable (also possible in .env). Or, overwrite any FLASK_ENV setting by passing an env in directly (useful for testing for instance).
A path to a config file can be passed in (otherwise a config file will be searched in the home or instance directories).
Also, a list of plugins can be set. Usually this works as a config setting, but this is useful for automated testing.
data.models
- exception flexmeasures.data.models.ModelException
utils
Utilities for the FlexMeasures project.
utils.calculations
Calculations
- flexmeasures.utils.calculations.integrate_time_series(s: pandas.core.series.Series, s0: float, decimal_precision: Optional[int] = None) pandas.core.series.Series
Integrate time series of length n and closed=”left” (representing a flow) to a time series of length n+1 and closed=”both” (representing a stock), given a starting stock s0. The unit of time is hours: i.e. the stock unit is flow unit times hours (e.g. a flow in kW becomes a stock in kWh). Optionally, set a decimal precision to round off the results (useful for tests failing over machine precision).
>>> s = pd.Series([1, 2, 3, 4], index=pd.date_range(datetime(2001, 1, 1, 5), datetime(2001, 1, 1, 6), freq=timedelta(minutes=15), closed="left")) >>> integrate_time_series(s, 10) 2001-01-01 05:00:00 10.00 2001-01-01 05:15:00 10.25 2001-01-01 05:30:00 10.75 2001-01-01 05:45:00 11.50 2001-01-01 06:00:00 12.50 Freq: D, dtype: float64
>>> s = pd.Series([1, 2, 3, 4], index=pd.date_range(datetime(2001, 1, 1, 5), datetime(2001, 1, 1, 7), freq=timedelta(minutes=30), closed="left")) >>> integrate_time_series(s, 10) 2001-01-01 05:00:00 10.0 2001-01-01 05:30:00 10.5 2001-01-01 06:00:00 11.5 2001-01-01 06:30:00 13.0 2001-01-01 07:00:00 15.0 dtype: float64
utils.time_utils
- flexmeasures.utils.time_utils.as_server_time(dt: datetime.datetime) datetime.datetime
The datetime represented in the timezone of the FlexMeasures platform. If dt is naive, we assume it is UTC time.
- flexmeasures.utils.time_utils.decide_resolution(start: Optional[datetime.datetime], end: Optional[datetime.datetime]) str
Decide on a practical resolution given the length of the selected time period. Useful for querying or plotting.
- flexmeasures.utils.time_utils.ensure_local_timezone(dt: Union[pandas._libs.tslibs.timestamps.Timestamp, datetime.datetime], tz_name: str = 'Europe/Amsterdam') Union[pandas._libs.tslibs.timestamps.Timestamp, datetime.datetime]
If no timezone is given, assume the datetime is in the given timezone and make it explicit. Otherwise, if a timezone is given, convert to that timezone.
- flexmeasures.utils.time_utils.forecast_horizons_for(resolution: Union[str, datetime.timedelta]) Union[List[str], List[datetime.timedelta]]
Return a list of horizons that are supported per resolution. Return values or of the same type as the input.
- flexmeasures.utils.time_utils.freq_label_to_human_readable_label(freq_label: str) str
Translate pandas frequency labels to human-readable labels.
- flexmeasures.utils.time_utils.get_most_recent_clocktime_window(window_size_in_minutes: int, now: Optional[datetime.datetime] = None) Tuple[datetime.datetime, datetime.datetime]
Calculate a recent time window, returning a start and end minute so that a full hour can be filled with such windows, e.g.:
Calling this function at 15:01:xx with window size 5 -> (14:55:00, 15:00:00) Calling this function at 03:36:xx with window size 15 -> (03:15:00, 03:30:00)
window_size_in_minutes is assumed to > 0 and < = 60, and a divisor of 60 (1, 2, …, 30, 60).
If now is not given, the current server time is used. if now / the current time lies within a boundary minute (e.g. 15 when window_size_in_minutes=5), then the window is not deemed over and the previous one is returned (in this case, [5, 10])
Returns two datetime objects. They’ll be in the timezone (if given) of the now parameter, or in the server timezone (see FLEXMEASURES_TIMEZONE setting).
- flexmeasures.utils.time_utils.get_timezone(of_user=False) pytz.tzinfo.BaseTzInfo
Return the FlexMeasures timezone, or if desired try to return the timezone of the current user.
- flexmeasures.utils.time_utils.localized_datetime(dt: datetime.datetime) datetime.datetime
Localise a datetime to the timezone of the FlexMeasures platform. Note: this will change nothing but the tzinfo field.
- flexmeasures.utils.time_utils.localized_datetime_str(dt: datetime.datetime, dt_format: str = '%Y-%m-%d %I:%M %p') str
Localise a datetime to the timezone of the FlexMeasures platform. If no datetime is passed in, use server_now() as basis.
Hint: This can be set as a jinja filter, so we can display local time in the app, e.g.: app.jinja_env.filters[‘localized_datetime’] = localized_datetime_str
- flexmeasures.utils.time_utils.naive_utc_from(dt: datetime.datetime) datetime.datetime
Return a naive datetime, that is localised to UTC if it has a timezone. If dt is naive, we assume it is already in UTC time.
- flexmeasures.utils.time_utils.naturalized_datetime_str(dt: Optional[datetime.datetime], now: Optional[datetime.datetime] = None) str
Naturalise a datetime object (into a human-friendly string). The dt parameter (as well as the now parameter if you use it) can be either naive or tz-aware. We assume UTC in the naive case.
We use the the humanize library to generate a human-friendly string. If dt is not longer ago than 24 hours, we use humanize.naturaltime (e.g. “3 hours ago”), otherwise humanize.naturaldate (e.g. “one week ago”)
Hint: This can be set as a jinja filter, so we can display local time in the app, e.g.: app.jinja_env.filters[‘naturalized_datetime’] = naturalized_datetime_str
- flexmeasures.utils.time_utils.resolution_to_hour_factor(resolution: Union[str, datetime.timedelta]) float
Return the factor with which a value needs to be multiplied in order to get the value per hour, e.g. 10 MW at a resolution of 15min are 2.5 MWh per time step.
- Parameters
resolution – timedelta or pandas offset such as “15T” or “1H”
- flexmeasures.utils.time_utils.server_now() datetime.datetime
The current time of the FlexMeasures platform. UTC time, localized to the FlexMeasures timezone.
- flexmeasures.utils.time_utils.tz_index_naively(data: Union[pandas.core.frame.DataFrame, pandas.core.series.Series, pandas.core.indexes.datetimes.DatetimeIndex]) Union[pandas.core.frame.DataFrame, pandas.core.series.Series, pandas.core.indexes.datetimes.DatetimeIndex]
Turn any DatetimeIndex into a tz-naive one, then return. Useful for bokeh, for instance.
ui.utils.view_utils
ui.utils.plotting_utils
- flexmeasures.ui.utils.plotting_utils.build_palette() Tuple[List[str], str, str]
Choose a color palette, and also single out our primary, forecasting and scheduling colors
- flexmeasures.ui.utils.plotting_utils.compute_legend_height(legend) float
Adapted from: https://github.com/bokeh/bokeh/blob/master/bokehjs/src/lib/models/annotations/legend.ts
- flexmeasures.ui.utils.plotting_utils.create_graph(data: pandas.core.frame.DataFrame, unit: str = 'Some unit', title: str = 'A plot', x_label: str = 'X', y_label: str = 'Y', legend_location: Union[str, Tuple[float, float]] = 'top_right', legend_labels: Tuple[str, Optional[str], Optional[str]] = ('Actual', 'Forecast', 'Schedules'), x_range: Optional[bokeh.models.ranges.Range1d] = None, forecasts: Optional[pandas.core.frame.DataFrame] = None, schedules: Optional[pandas.core.frame.DataFrame] = None, show_y_floats: bool = False, non_negative_only: bool = False, tools: Optional[List[str]] = None, sizing_mode: str = 'scale_width') bokeh.plotting.figure.Figure
Create a Bokeh graph. As of now, assumes x data is datetimes and y data is numeric. The former is not set in stone.
- Parameters
data – the actual data. Expects column name “event_value” and optional “belief_horizon” and “source” columns.
unit – the (physical) unit of the data
title – Title of the graph
x_label – x axis label
y_label – y axis label
legend_location – location of the legend
legend_labels – labels for the legend items
x_range – values for x axis. If None, taken from series index.
forecasts – forecasts of the data. Expects column names “event_value”, “yhat_upper” and “yhat_lower”.
schedules – scheduled data. Expects column name “event_value”.
hover_tool – Bokeh hover tool, if required
show_y_floats – if True, y axis will show floating numbers (defaults False, will be True if y values are < 2)
non_negative_only – whether or not the data can only be non-negative
tools – some tools for the plot, which defaults to [“box_zoom”, “reset”, “save”].
- Returns
a Bokeh Figure
- flexmeasures.ui.utils.plotting_utils.create_hover_tool(y_unit: str, resolution: datetime.timedelta, as_beliefs: bool = False) bokeh.models.tools.HoverTool
Describe behaviour of default tooltips (we could also return html for custom tooltips)
Uses from_py_func, a deprecated function since bokeh==1.1 https://docs.bokeh.org/en/latest/docs/releases.html?highlight=from_py_func
- flexmeasures.ui.utils.plotting_utils.decide_plot_resolution(data: pandas.core.frame.DataFrame) datetime.timedelta
Finding out which resolution to use: prefer resolution in data, otherwise from session (which is based on the session’s time period)
- flexmeasures.ui.utils.plotting_utils.get_latest_power_as_plot(asset: flexmeasures.data.models.assets.Asset, small: bool = False) Tuple[str, str]
Create a plot of an asset’s latest power measurement as an embeddable html string (incl. javascript). First returned string is the measurement time, second string is the html string.
- flexmeasures.ui.utils.plotting_utils.highlight(fig: bokeh.plotting.figure.Figure, x_start: Any, x_end: Any, color: str = '#FF3936', redirect_to: Optional[str] = None)
Add a box highlight to an area above the x axis. If a redirection URL is given, it can open the URL on double-click (this assumes datetimes are used on x axis!). It will pass the year, month, day, hour and minute as parameters to the URL.
- flexmeasures.ui.utils.plotting_utils.make_datasource_from(data: pandas.core.frame.DataFrame, resolution: datetime.timedelta) bokeh.models.sources.ColumnDataSource
Make a bokeh data source, which is for instance useful for the hover tool.
- flexmeasures.ui.utils.plotting_utils.make_range(index: pandas.core.indexes.datetimes.DatetimeIndex, other_index: Optional[pandas.core.indexes.datetimes.DatetimeIndex] = None) Optional[bokeh.models.ranges.Range1d]
Make a 1D range of values from a datetime index or two. Useful to share axis among Bokeh Figures.
- flexmeasures.ui.utils.plotting_utils.replace_source_with_label(data: pandas.core.frame.DataFrame) pandas.core.frame.DataFrame
Column “source” is dropped, and column “label” is created. The former column should contain DataSource objects, while the latter will contain only strings.
- flexmeasures.ui.utils.plotting_utils.separate_legend(fig: bokeh.plotting.figure.Figure, orientation: str = 'vertical') bokeh.plotting.figure.Figure
Cuts legend out of fig and returns a separate legend (as a Figure object). Click policy doesn’t work on the new legend. Requires bokeh==1.0.2 for solution from https://groups.google.com/a/continuum.io/forum/#!topic/bokeh/BJRhWlnmhWU Open feature request to share a legend across plots: https://github.com/bokeh/bokeh/issues/7607
ui.views
- flexmeasures.ui.views.dashboard.dashboard_view()
Dashboard view. This is the default landing page for the platform user. It shows a map with the location and status of all of the user’s assets, as well as a breakdown of the asset types in the user’s portfolio. Assets for which the platform has identified upcoming balancing opportunities are highlighted.
- flexmeasures.ui.views.portfolio.data_or_zeroes(df: pandas.core.frame.DataFrame, start, end, resolution) pandas.core.frame.DataFrame
Making really sure we have the structure to let the plots not fail
- flexmeasures.ui.views.portfolio.get_flex_action_hour(h: int) datetime.datetime
get the next hour from now on
- flexmeasures.ui.views.portfolio.portfolio_view()
Portfolio view. By default, this page shows live results (production, consumption and market data) from the user’s portfolio. Time windows for which the platform has identified upcoming balancing opportunities are highlighted. The page can also be used to navigate historical results.
- flexmeasures.ui.views.portfolio.rename_event_value_column_to_resource_name(df_dict: Dict[str, pandas.core.frame.DataFrame]) Dict[str, pandas.core.frame.DataFrame]
Replace the column name “event_source” with the resource name, for each resource in the dictionary.
- flexmeasures.ui.views.portfolio.stack_df(df: pandas.core.frame.DataFrame) pandas.core.frame.DataFrame
Stack columns of df cumulatively, include bottom
- flexmeasures.ui.views.control.control_view()
Control view. This page lists balancing opportunities for a selected time window. The user can place manual orders or choose to automate the ordering process.
- flexmeasures.ui.views.analytics.analytics_data_view(content, content_type)
Analytics view as above, but here we only download data. Content can be either source or metrics. Content-type can be either CSV or JSON.
- flexmeasures.ui.views.analytics.analytics_view()
Analytics view. Here, four plots (consumption/generation, weather, prices and a profit/loss calculation) and a table of metrics data are prepared. This view allows to select a resource name, from which a models.Resource object can be made. The resource name is kept in the session. Based on the resource, plots and table are labelled appropriately.
- flexmeasures.ui.views.analytics.determine_resolution(data: pandas.core.frame.DataFrame, forecasts: Optional[pandas.core.frame.DataFrame] = None, schedules: Optional[pandas.core.frame.DataFrame] = None) str
Determine the resolution to be displayed under the plot. We try to get it from the DataFrame’s meta data, or guess from the actual data. Lastly, we try the session. If nothing can be found this way, the resulting string is “?”
- flexmeasures.ui.views.analytics.filter_for_past_data(data)
Make sure we only show past data, useful for demo mode
- flexmeasures.ui.views.analytics.filter_forecasts_for_limited_time_window(data)
Show forecasts only up to a limited horizon
- flexmeasures.ui.views.analytics.get_data_and_metrics(query_window: Tuple[datetime.datetime, datetime.datetime], resolution: str, show_consumption_as_positive: bool, showing_individual_traces_for: str, selected_resource: flexmeasures.data.services.resources.Resource, selected_market_sensor: flexmeasures.data.models.time_series.Sensor, selected_sensor_type, assets) Tuple[Dict[str, pandas.core.frame.DataFrame], Dict[str, float], str, flexmeasures.data.models.time_series.Sensor]
Getting data and calculating metrics for them
- flexmeasures.ui.views.analytics.make_power_figure(resource_display_name: str, data: pandas.core.frame.DataFrame, forecast_data: Optional[pandas.core.frame.DataFrame], schedule_data: Optional[pandas.core.frame.DataFrame], show_consumption_as_positive: bool, shared_x_range: bokeh.models.ranges.Range1d, tools: Optional[List[str]] = None, sizing_mode='scale_width') bokeh.plotting.figure.Figure
Make a bokeh figure for power consumption or generation
- flexmeasures.ui.views.analytics.make_prices_figure(data: pandas.core.frame.DataFrame, forecast_data: Union[None, pandas.core.frame.DataFrame], shared_x_range: bokeh.models.ranges.Range1d, selected_market: flexmeasures.data.models.markets.Market, tools: Optional[List[str]] = None, sizing_mode='scale_width') bokeh.plotting.figure.Figure
Make a bokeh figure for price data
- flexmeasures.ui.views.analytics.make_revenues_costs_figure(resource_display_name: str, data: pandas.core.frame.DataFrame, forecast_data: pandas.core.frame.DataFrame, show_consumption_as_positive: bool, shared_x_range: bokeh.models.ranges.Range1d, selected_market: flexmeasures.data.models.markets.Market, tools: Optional[List[str]] = None, sizing_mode='scale_width') bokeh.plotting.figure.Figure
Make a bokeh figure for revenues / costs data
- flexmeasures.ui.views.analytics.make_weather_figure(selected_resource: flexmeasures.data.services.resources.Resource, data: pandas.core.frame.DataFrame, forecast_data: Union[None, pandas.core.frame.DataFrame], shared_x_range: bokeh.models.ranges.Range1d, weather_sensor: flexmeasures.data.models.time_series.Sensor, tools: Optional[List[str]] = None, sizing_mode='scale_width') bokeh.plotting.figure.Figure
Make a bokeh figure for weather data