Storage FAQ

Extraction is just one step of the data journey. Our online storage offers a waystation to help your vital data get where it's going. Practice exporting, publishing, and connecting to data sources so it can drive your big decisions.

What is a collection?
Collections are table where an agent's data is written and stored. Data in collections can be collected by agents or manually uploaded by the user. Each line or row in a collection is referred to as an item.

Where can I find storage?
An overview of your storage is found on the navigation bar of your Web Console. Select to be brought to your peak storage statistics.

How is my storage calculated?
Our system records your used storage once a day. Your monthly peak storage equals the highest daily storage on record. For example, if your account storage hits 112 GB on February 14th, your peak storage for the month of February is 112 GB.

Can I change the time frame of the storage statistics?
See a breakdown by day, week, month and quarter, by selecting the drop-down in the far right-hand corner of the screen.

How do I preview and download my statistics?
Preview statistics by selecting an item in the list. Select to download statistic for the selected item .

Statistics fields on storage:

• All metrics are displayed in megabytes.
• Database refers to text data.
• AgentDefinitions refers to the XML file of the agent.
• Files and Packages refers to any files that have been downloaded during harvesting.

What makes up my storage?

• Items: The rows within a collection of data - either uploaded by the user to an input collection or collected during harvesting within an agent collection.

• Job History: Jobs ran on the agent or collection within the last year.
• Images and Files: Also referred to as Web files. Long term storage of files downloaded during harvesting including images.

• File Packages: Short term compressed storage of downloaded images and files used for exporting and publishing. If cleared, this storage can be brought back by repackaging files in the collection.

• Item history and bookmarks: This is available when store item history is enabled. When item history is enabled, all data and files are kept from previous runs in organization units called "bookmarks". Bookmarks are one complete gathering of the data, i.e. if an agent has been run and has multiple jobs, all data will be stored in a single bookmark. Clearing item history and bookmarks deletes all history and items for the collection.

How is storage billed?
$10 per GB will be charged if your monthly peak storage exceeds your account limit. Any overages accumulated the months before will be added to the next month's invoice. Billing will begin February 1st, 2020.

How do I clear storage?
Select Collections from the navigation bar of the web console. This will bring you to a list of collections in your account.
Select a specific collection > Click on More > Clear collection data.

Difference between File Packages and Images and Files?

Clearing Images and Files will permanently delete the files and images while clearing File Packages will free up temporary storage until the agent has been run again and or the files are repackaged.

How do I download my files?
To download the files and images collected, go to your agent or collection> select > Export > Select the Images & Files tab > Click Package Files to submit your request to repackage any images or files > Select Download.

If you are republishing files then the files and images will be packaged as part of the publishing process.

Are my files packaged and stored even when I publish the data somewhere else?
When the data is being published, the files/images are packaged and sent to the publishing source. There is a setting that controls if the files packages are stored in the Mozenda Cloud. This setting is found under Harvesting settings in Behavior. Set this on an agent basis or an account basis, for all agents moving forward.

Single Agent Setting Verification:
Go to the agent > select > Harvesting Settings > click Behavior > select Do not create image/file packages.

Global Settings:
From the Summary Page, select Account Settings > Change Harvesting Setting defaults, then check “Do not create image/file packages”.

From that point forward any agent built with file/images will not be packaged.

The only reason why you would need to package files is if you are downloading the files manually from the web console. We recommend that this is selected for all agents.

In sequences, can I set a step to delete my collections?
Yes, depending on your licensing, a step can be added to sequences, or API, to automatically clear a collection. We recommend setting a step to publish the data and clear the collection afterward. Sequences are available for enterprise licenses.

How can I automatically clear my storage?
The best way to automatically clear storage and collections is to use the API, implement a sequence, or clear manually. Access to the API is available for all enterprise licenses and professional subscriptions. Sequences are available for enterprise licenses.