Tips - Database Size

(Also See: Database Backup And Recovery) (Also See: Tip: Database Utilities)

You can potential reduce the size of your database by following the steps below. By reducing the size of your database, you will also reduce the time it takes to verify and backup your database.

First go to “Control: Data Management: Delete/Rename: Historical Data”, and delete all historical data that you do not need with this interface. For example you can delete all daily data on file dated before a specified date using this facility. There are now shortcuts to quickly delete all historical data (or to delete all intraday or daily, weekly, monthly data). You will find these options in the "Control: Database Utilities" menu. The options are "Initialize Intraday Database", "Initialize D/W/M Database", or "Initialize Historical Database".

Now, go to "Setup: Preferences: Historical". In the "Default Retention Settings", specify the number of days for which you want to retain intraday and daily data, as well as the number of year for which you want to retain weekly and monthly data. Keep these numbers as small as possible to reflect only the amount of data required for your analysis. Also, make sure the "Compact" checkbox is checked. It is recommended that you specify "Compact ticks older than 0 days into 1 minute bars". With this setup, all ticks will be combined into 1-minute bars at posting time each day at the end of the session. Active instruments can generate tens of thousands of ticks (trades) per day. Compacting these ticks into 390 or so 1-minute bars can really keep the database size down. Back to the Historical Retention Preferences, in the section below "Default Retention Settings" is an area where you can override the default settings for individual instruments if you like. Just choose the instrument on the left, then specify the override settings on the right. Use this facility to specify longer retention periods for special cases.

Go to “Control: Data Management: Delete/Rename: Charts” and delete any charts or other objects that you do not use/need, including quotepages, portfolios, scans, custom indicators, presets, custom columns, quick quotes, layouts, trading signals, and trading systems.

Open the “.All Symbols” quotepage and delete (with red trash can) all symbols you do not need.

Open the ".Intraday" quotepage. This quotepage contains all the instruments that are marked for intraday. Symbols that are marked for intraday are monitored tick by tick on the data feed. If there are any instruments on this list for which you do not need intra-day monitoring, remove them from the .Intraday list using the green trash can button. First click to select the row, then click to remove the ticker from the page. Removing an instrument from the .Intraday page causes it to be unmarked for intra-day monitoring.

Go to "Setup: Preferences: News". If you're not concerned with news at all within Investor/RT, "Purge", "Delete", and "Empty" the news, and then check the “Turn OFF” checkbox and both "Purge" checkboxes. This will tell Investor/RT to stop tracking and maintaining any news stories that are coming from your data service. This can cut down quite a bit on database size.

Once you have completed these steps, go to “Control: Database Utilities”, and perform both “Compress” utilities. First, run the "Compress Intraday Database" utility, and then the "Compress D/W/M Database" utility (compresses daily, weekly, and monthly data).

After compressing, perform a database backup using "Control: Database Utilities: Backup Database". After the backup is completed, use "File: Open: Backup Report" to review the size of the database and each "file". In particular compare the size of the files named hist.d01, tick.d01 and news.d01 to their former sizes. These three files contain historical data and news items and are typically the largest of the files in the data_f folder.