Writing Scans

Writing Scans

Both advanced scans and technical alerts use a special language (which we call scan syntax) to communicate with the Scan Engine. It takes some time to master, but provides a great deal of flexibility. In this article, we'll equip you with the basic building blocks you need to write effective scans.

What is a Scan?

A scan is a series of tests that are performed by the Scan Engine, to see which stocks meet all your technical requirements. Stocks that pass the first test (meet the first technical requirement) move on to the second test. When all of the tests in your scan have been conducted, the stocks that have passed all the tests are determined to meet your technical requirements and are included in your scan results.

Scan Clauses

Scans are divided into scan clauses, and each clause gives the Scan Engine instructions about how to test for a particular technical requirement.

A clause has two expressions that are compared with each other using a comparison operator. The clause is enclosed in square brackets. Here's a very simple example of a clause:

[close > 5]

The two expressions in this clause are “close” (the closing value of the stock), and the value “5”. The comparison operator is the “greater than” symbol. So, this clause is testing to see if the closing value of each stock is more than $5.

If the statement is true for a stock, then that stock passes the test. So, all stocks with a closing value of more than $5 would be passed through to the next test. All stocks with a closing value less than or equal to $5 would be excluded from the scan results.

Comparison Operators

We've already seen the “greater than” comparison operator, but there are several other operators that can be used in your scan clauses.

Some are usually used for text comparisons:

  • is
  • is not
  • contains
  • not contains
  • starts with
  • ends with

Some are usually used for number comparisons:

  • = (equals)
  • < (less than)
  • <= (less than or equal to)
  • > (greater than)
  • >= (greater than or equal to)
  • != (not equal to)
  • x (crosses above)

Note that ”=” and “is” can be used interchangeably in most clauses. The same is true for ”!= ” and “is not”. The Scan Engine will interpret both of these scans the same: ”[type = stock]” and ”[type is stock]”.

Crosses Above Operator

The Crosses Above operator needs some additional explanation. This operator is frequently used when testing for a technical signal. It basically means that the expression on the left has just crossed above the expression on the right: yesterday it was below, and today it is above. Here's a very simple example:

[close x 5]

This clause is testing for stocks whose closing value has just crossed above $5 today. This means that yesterday the stock's value was below $5, and today it is above $5.

Note: we do not have a “Crosses Below” operator, because you can easily accomplish the same thing by using the “Crosses Above” operator with the positions of the two expressions switched. Building on our example above, if you wanted to test for the closing price crossing below $5, you would use the following clause:

[5 x close]

If $5 has just crossed above the close, then that means that the close has just crossed below $5.

Learn More: Writing Crossover Scans

Scan Expressions

We've seen one example of an expression (close), but there are so many more expressions you can use in your scans. You can browse through the Scan Components dropdown menus in the Advanced Scan Workbench to get a good idea of the expressions that are available to you, and be sure to check out our Scan Syntax Reference for all the details on adding those expressions to your scans.

The expressions are grouped into these general categories in the Scan Components area:

  • Ticker Properties - security type, symbol, name, country, exchange, and more.
  • Price, Volume, and SCTRs - standard price and volume measurements (OHLC values, all time highs, volume, liquidity, etc.), plus SCTR rankings.
  • Technical Indicators (and Functions) - several popular indicators, from MACD to RSI to Stochastics. Also includes standard arithmetic functions, such as min/max and percent change.
  • Candlestick Patterns - common bullish and bearish candlestick patterns, such as hammers or morning stars. Also includes Elder Bar and Ichimoku Cloud patterns.
  • Candlestick Building Blocks - candlestick characteristics that are used to build candlestick patterns - everything from dojis to spinning tops.
  • P&F Patterns - standard bullish and bearish P&F patterns, such as bullish triangles and double bottom breakouts
  • Predefined Scans - all the technical indicators found on our Predefined Scans page, from MACD crossovers to 52-week highs/lows.
  • Indexes and ETFs - limit your results to stocks included in a particular index, ETFS that are not inverse/leveraged, and more
  • ChartLists - limit your results to stocks in one of your ChartLists
  • Sectors and Industries - limit your results to stocks that are (or are not) in a particular sector or industry

Expression Parameters

Some expressions require configuration parameters. For example, if you are using the RSI indicator, you need to tell the Scan Engine how many periods you want to use to calculate the RSI value. These parameters appear in parentheses right after the indicator name. For example, the clause ”[RSI(14) > 0]” says you want tickers where the 14-period RSI value is greater than zero.

When there are multiple parameters for a single indicator, they are separated by commas, e.g. ”[MACD Line(12,26,9) > 0]”.

The Technical Indicators section of the Scan Syntax Reference explains what the parameters are for each indicator.

Remember, we only use parentheses to designate the parameters for indicators. Square brackets are used to show the start and end of scan clauses.

Modifying Expressions

So far we have looked only at clauses with very simple expressions. There are a number of ways to modify expressions to make them more specific.

Date Periods for Expressions

By default, expressions are referring to daily data, but it is relatively simple to change an expression to use weekly or monthly data instead. Just add the word 'weekly' or 'monthly' in front of the expression. For example:

[weekly MACD Line(12,26,9) x 0]
[monthly RSI(14) > 0]
[daily close > weekly close]

Note that you can even mix and match date periods within a single clause, as in the third example above.

Date Modifiers

In addition to changing the date offset for the entire scan using the Starting Date Adjuster, you can also change the date offset for individual expressions by adding a date modifier at the beginning of the expression.

By default, today's data is used, but you can use any of the following modifiers:

today's
yesterday's
n days ago (e.g. 3 days ago)
this week's
last week's
n weeks ago (e.g. 2 weeks ago)
this month's
last month's
n months ago (e.g. 4 months ago)

These date modifiers refer to trading days, not calendar days; so a clause with a “5 days ago” modifier is looking at data from 5 trading days ago - essentially a week ago.

Here are a few examples of clauses with date modifiers:

[4 days ago close > 38]
[this week's high > 1 week ago high]
[monthly low < last month's low]

Note that you can mix and match date modifiers within a single clause, as in the second and third examples above.

Arithmetic Operators

Standard arithmetic operators are available for use in any expression:

  • + (Plus)
  • - (Minus)
  • * (Multiplied By)
  • / (Divided By)

Here are a couple of the most commonly encountered uses of arithmetic operators:

[volume > yesterday's volume * 1.05]
[close> [[high + low] / 2]]

The first example scans for stocks where today's volume is at least 5% higher than yesterday's volume. The second example scans for stocks where the closing price is in the upper half of the day's range.

The arithmetic operators are processed from left to right, but items enclosed between square brackets will be evaluated as a single unit. For example:

[close > low + [0.90*[high - low]]]

The brackets around “high - low” ensure that this subtraction will be done before it is multiplied by 0.90. Similarly, the brackets around “0.90*[high-low]” ensure that all of that math will be done before the result is added to the low.

Remember, we only use parentheses to designate the parameters for indicators. Square brackets are used to show the Scan Engine what order to process arithmetic operators in.

Arithmetic Functions

There are several standard arithmetic functions that can be applied to your scan expressions:

  • Min/Max - these two functions find the minimum or maximum value in a set of values. For example, [today's close> yesterday's max(255,close)] scans for stocks where today's close is higher than the highest close over the previous 255 days. Learn more in our Support Center article on Writing Min/Max Scans.
  • AbsVal - this function finds the absolute value of a value. It is often used with oscillators, to find values that have strayed far from the center line in one direction or another. For example, [AbsVal(MACD Hist(12,26,9)) > 3] scans for stocks where the MACD Histogram value is either above 3, or below -3.
  • PctChange - this function finds the percentage change of a value over a specified time period. For example, [PctChange(10,close) > 15] scans for stocks where the value has changed by at least 15% over the previous 10 trading days.
  • PctDiff - this function finds the percentage difference between two values. For example, [PctDiff(close,sma(50,close)) < 5] scans for stocks where today's closing value is within 5% of its 50-day average.

For more details on using these functions, see the Technical Functions section of our Scan Syntax Reference.

Logical Operators

It's pretty rare for a scan to only have one clause. Usually you are testing for multiple technical conditions and signals. The logical operators “AND” and “OR” are used to string multiple clauses together into a scan.

The AND Operator

If you use the “AND” operator to add multiple clauses to your scan, that means that all the clauses need to be true. Stocks must pass the first test, AND they must pass the second test. Generally, you want your scan to pass all the tests (otherwise, why include a test in your scan?), so “AND” is the default operator used to connect clauses.

For example, take the following scan:

[type = stock] 
AND [country = US] 
AND [close >10]

First, each ticker symbol needs to pass the first test: is it a stock? Only securities that are stocks will pass and go on to the second test: is US the country? Since the second clause is connected with “AND”, we know that the stock also must pass the second test. Only US stocks will pass both the first and second tests, and go on to the third test: is today's closing price above $10? Again, since the clause is connected with “AND”, we know that the stock must pass this third test too. Only US stocks with a close above $10 will pass all three tests and be included in the scan results.

The OR Operator

There is one special situation where a ticker symbol doesn't have to pass all the tests in the scan; in some cases, there are multiple similar clauses that could meet the requirement you're looking for. If any one of those clauses is true, then you want to return that ticker symbol in your scan results. For example, let's say you want to return all stocks that either trade on the Nasdaq OR on the NYSE. It doesn't have to trade on both exchanges; as long as it trades on one of them, you want the stock to be part of your search results.

In that case, we use the “OR” operator to define the list of clauses that could fulfill this requirement. In our example above, it might look like this:

[type = stock] 
AND [[exchange = Nasdaq] OR [exchange = NYSE]]

In this case, we're looking for stocks that trade on either the Nasdaq or the NYSE exchange.

Notice that in addition to the square brackets around each clause, we also have an additional set of brackets that encompasses the entire list of “OR” clauses that we are testing for. These brackets are very important, because they make it clear to the scan engine which clauses are included in your “OR” list, and which ones aren't. Without them, you may get unexpected results. For more on why this is so important, check out the OR Clause section of our Troubleshooting Scans article in the Support Center.

With these extra brackets, we're basically telling the scan engine to run the first test (is it a stock?), and if it passes that test, then move on to the second test: does it trade on the Nasdaq or the NYSE? Those extra brackets make it clear that the second and third clauses are all part of a single multi-part test.

Note: we recommend putting each “test” on a separate line in your scan. It makes no difference to the Scan Engine, but it definitely makes it easier for humans to read.

Here's a more complicated example using OR:

[type = stock] 
AND [[MACD Line(12,26,9) x 0] OR [ROC(14) x 0] OR [CMF(20) x 0]]

In this example, there are three possible conditions that the stock could meet to pass the second test: either the MACD, ROC, or CMF could cross above zero. All three of these oscillator crossovers indicate a change from negative to positive momentum, so if a stock has any one of these crossovers, it indicates a bullish move. It is not necessary for all three crossovers to happen for the stock at the same time; as long as at least one crossover happens, the stock passes the test.

OR clauses can provide a lot of flexibility and power to your scans, as long as they are used with caution. The important takeaway is to always put an additional set of square brackets enclosing all of the OR clauses that should be part of that test.

Rank By Clauses

You can optionally add a line to the very end of your scan that controls the order in which your scan results are displayed. That line must start with “RANK BY”, followed by the expression that you want your results sorted by. For example:

RANK BY [close]

This example would sort the scan results in descending order of current closing price.

There are a few rules for Rank By clauses:

  • There can only be one Rank By line per scan
  • It must be the last line of the scan
  • You can add “ascending” to the end of the clause to reverse the order of the scan results, e.g. “RANK BY [MACD Hist(12,26,9)] ASCENDING”

Note: the value of the Rank By expression is also added to the right side of the scan results table. This allows you to examine the value of any expression in your scan directly, simply by making that expression the Rank By expression as well.

Comments

It is possible to have the Scan Engine ignore portions of the scan text by adding either the '#' character or two forward slashes ('//') in your scan text. The Scan Engine will ignore all the text from that point to the end of the current line.

Comments are very useful in two ways. The first use is to document a particularly complex scan which contains a lot of clauses. You can add useful notes for yourself, without affecting the Scan Engine at all.

For example:

// My ETF scan.
// July 20, 2014
[type = stock]
# only get ETFs > 100
AND [close > 100]  // Let's get the expensive ones!
AND [group is ETF]

When the Scan Engine processes this, it only sees:

[type = stock]
AND [close > 100]
AND [group is ETF]

The Scan Engine ignores all the comment text.

The other use for comments is to selectively add or remove clauses and view the results. Here is an example of toggling between two clauses within a scan:

[type = stock]
AND [open > 25]

// AND [group is ENERGYSECTOR]
AND [group is FINANCIALSECTOR]

In this example, we can quickly switch between scanning the Financial or Energy sectors by commenting out one of the 'group is' clauses. Being able to quickly add or remove clauses using comments is very helpful for debugging scans. For more information on this usage, please see our Troubleshooting Scans article in the Support Center.

Conclusions

Scanning is one of the most difficult parts of our site to master, but with a little trial and error, you'll soon be writing complex scans. When writing scans in the Advanced Scan Workbench, remember to click the Check Syntax button as you make changes, to make sure you uncover any problems with your scan syntax early. If you're not getting errors, but also not getting the results you expect, be sure to check out our Troubleshooting Scans article in the Support Center for more troubleshooting tips.

Additional Resources

Videos