fredcli batch submit

Submit the current directory as a FRED model.

Synopsis

fredcli batch submit

Execute the model in the current directory in FRED Cloud

fredcli batch submit [options]

Options

-s, --size <size>

The size to use for the simulation environment (see –help-size)

Options

tiny | small | default | medium | large | massive | small2 | default2 | medium2 | large2 | xlarge2 | small-mem | default-mem | medium-mem | large-mem | massive-mem

--config-file

Tell fredcli to expect config file.

--container-log-level <container_log_level>

The log level used in the container running the simulation. [default: error]

Options

debug | info | warning | error | critical

-f, --file <file>

Designate the config file to use.

Default

fredconfig.json

--fred-version <fred_version>

Designate the version of FRED to use. [default: latest]

-l, --label <label>

Label the batch. Can have multiple labels by using this option multiple times. i.e. –label Label1 –label Label2

-k, --kill <kill>

Amount of time to wait before killing the batch. Can be hours(h) or days(d). i.e. -k 24h is 24 hours, -k 2d is two days. [default:1d]

--output <output>

The output format for this function.

Default

text

Options

text | json | yaml

--spot

[Alpha] This parameter is for scheduling a batch to run on a Spot compute resources, instead of On-Demand. * WARNING * This feature can lead to an unexpected termination of a batch, requiring manual intervention. This feature is not recommended for long running batches

Default

False

--validate-only

Validate the configuration file without submitting the batch.

--help-size

Show details on the available size options

Description

This command submits the FRED model in the current working directory to FRED Cloud for execution. By default, the command will look for an execute script called fred-cloud-execute. It is an error if this script is not found and the --config-file option is not specified.

The entire directory is packaged and uploaded to FRED Cloud, except for ignored files (see below), and the execute script is invoked within a Docker container running FRED. For information on how to create an execute script, see the section Batch Execution Scripts.

If the --config-file option is specified, then the command looks for a configuration file instead of an execute script. In this case, the file fredconfig.json is used by default. The -f or --file option can be used to override this default and specify a name for the configuration file. For details on the configuration file format, see Batch Configuration Files.

To verify a configuration file without submitting the model for processing, use the --validate-only option. The --config-file option is still required when using this flag.

The size of the container is specified by the --size option. The list of possible container sizes are shown in the following tables. The specific sizes available to you can be displayed on the command-line with the option --help-size. This command also displays cost information based on your organization. The specific containers available and cost per hour vary depending on your FRED Cloud configuration.

Standard Sizes

name

aws ami

cores

ram (GB)

tiny

r6gd.large

2

16

small

r6gd.xlarge

4

32

default

r6gd.2xlarge

8

64

medium

r6gd.4xlarge

16

128

large

r6gd.8xlarge

32

256

massive

r6gd.16xlarge

64

512

Double Sizes

name

aws ami

cores

ram (GB)

small2

x2gd.xlarge

4

64

default2

r5ad.2xlarge

8

128

medium2

x2gd.4xlarge

16

256

large2

x2gd.8xlarge

32

512

xlarge2

x2gd.16xlarge

64

1024

High Memory Sizes

name

aws ami

cores

ram (GB)

small-mem

x1e.xlarge

4

122

default-mem

x1e.2xlarge

8

244

medium-mem

x1e.4xlarge

16

488

large-mem

x1e.16xlarge

64

1952

massive-mem

x1e.32xlarge

128

3904

By default, the latest version of FRED is used to execute the model. Use the -fredVersion option to use a specific version of FRED.

You can alter the amount of log information produced within FRED Cloud using the --container-log-level option. This affects the amount of information available from the fredcli batch get-logs command. The possible log level values are as follows.

  • critical includes only critical errors in the logs

  • error includes any error in the logs, and is the default

  • warning includes both warning and error messages in the logs

  • info includes informational messages as well as warning and errors in the logs

  • debug includes detailed information about the processing of the batch in the logs, including information, warning, and error messages

You can label batches at submission by using the --label option. A batch can be assigned multiple labels by using the option repeatedly. For example if you would like to label a batch as “calibration” and “new-york” the command would be: fredcli batch submit --label calibration --label new-york. Labels can only contain alphanumeric characters, dashes(-), and underscores(_).

By default, FRED Cloud will wait up to 1 day (24 hours) for a batch to complete. After this day after submission has passed, the batch will be killed even if the batch only started processing recently. To extend (or reduce) the amount of time that FRED Cloud will wait for a batch to complete, use the -k or --kill option. This accepts a numeric value with a suffix of h for hours or d for days.

To ignore files or directories, a .fredcli-ignore file can be created in the model directory. Any files or directories listed in this file, using one item per line, will be ignored and not uploaded to FRED Cloud. By default, the .git directory and the .fredcli-ignore file are ignored, if they exist. Note that ignored items (files or directories) must be the actual name of the item to ignore; wildcards and other constructs are not supported in the ignore file at this time.

To execute the model on a spot compute resource, you can use the --spot CLI flag, or set the key environment.spot with a value of true in the batch config json file.

The final option available for this command is the --output option, with possible values text, json, or yaml. By default, the output option of text is used.

Examples

The following command runs the model in the current directory using the local fred-cloud-execute script in a tiny container in FRED Cloud.

$ fredcli batch submit -s tiny
Packaging model code.
Uploading model code.
Model code successfully uploaded.
Starting execution.
Your batch guid is cafdde0ea5bb4550be2242e7878032ea
To track the status of your batch use fredcli batch get-status cafdde0ea5bb4550be2242e7878032ea

Here is another example where the default configuration file fredconfig.json is used to execute the model with json output

$ fredcli batch submit --output json --config-file
{
"batch_id": "843fc3e4dca64e9897944788a60733c3"
}

See Also

Batch Configuration Files, Batch Execution Scripts, fredcli batch get-status, fredcli batch get-logs