Introduction to Advanced Scanning

Introduction to Advanced Scanning

Have you ever found a stock that has gone through the roof and thought “If only I had found that stock before it took off!”? We've all been there, some of us more often than we'd like!

No matter what the overall market is doing, there are always stocks out there doing interesting things such as making new highs, having pivotal crossovers or breaking out of old trading ranges. With thousands of stocks to choose from and hundreds of indicators and patterns available, trying to find the interesting ones is virtually impossible.

That means that you are missing these opportunities because you simply don't know they exist. There is just too much data to process without help.

That is the exact problem that the StockCharts Scan Engine solves. The Scan Engine will allow you to sift through thousands of stocks to spot these golden opportunities that you have been missing. This makes it one of the most powerful tools available to you on StockCharts. Taking the time to learn how to use the Scan Engine effectively may be one of the most profitable decisions you will ever make!

About These Tutorials

These tutorials will quickly show you everything you need to know in order to create and run technical scans successfully. In the process, you will learn how the Scan Engine works and you will develop the key skills necessary to use it effectively. Step-by-step we will develop a number of scans that demonstrate the key features of the Advanced Scan Workbench. We'll also go over some best practices which will make creating scans easier. And finally, we will show you where to get additional help and support as you develop your own custom scans.

You don't need to a super-experienced technical analyst to get a lot out of this document. The tutorials assume just a basic knowledge of common technical indicators and chart patterns. We suggest that if you come across an unfamiliar term, simply enter it into the “Search the Site” box on any StockCharts.com page. Chances are we already have a ChartSchool article with the answer to your question.

TIP: To get the most out of these tutorials we recommend you print this page out first so that you can refer to it as you follow along on your computer.

OK, let's get started…

We've talked about what the Scan Engine can do for us, but what exactly is the Scan Engine and how does it manage to find stocks that we are interested in? In the next section, we'll answer those questions and see what makes up the Scan Engine. We will also run our very first scan!

What Exactly is the Scan Engine?

The Scan Engine takes a vast amount of price data and applies filtering criteria that you provide to select only those stocks that you are interested in. Those results can then be displayed in a number of different formats for further analysis. The following illustration presents an overview of the various components of the scanning process:

scanengineflow

As you can see at the top of the diagram, data for thousands of stocks is contained in the Scan Engine. Users create search criteria using one of the two workbenches provided, and that criteria is then used to filter these millions of data points down into a result set. The result set is the list of all the stocks that satisfy your search criteria. Once we have a result set, we can move that data into a number of additional formats for further analysis. StockCharts provides two different workbenches for creating scans - the Standard Scan Workbench and the Advanced Scan Workbench. This tutorial focuses exclusively on the Advanced Scan Workbench because, as you will see, it provides much more flexibility.

Tutorial #1 - Running a Scan

OK, let's see how quick and easy it is to find some great performing stocks! Please try to complete each of the steps outlined below on your computer.

Step 1: Open a web browser window and go to StockCharts.com, then click on the Members tab at the top of the page.

Step 2: If you are not already logged in, enter your login information. After successfully logging in, go to the Advanced Scan Workbench by clicking on its link in the “Control Center” section at the top left of the page.

Step 3: Select the existing text in the Scan Criteria Box with your mouse and then delete it. Afterwards carefully enter the following:

[type = stock]
and [RSI(14) > 80]

Step 4: Click the 'Check Syntax' button to make sure there are no mistakes in your scan. If an error appears, check to make sure what you entered exactly matches what's in the box above.

Step 5: Click the 'Run Scan' button to send your scan to the Scan Engine.

Step 6: A new browser window (or a new tab in your current browser window) should appear with a results page that looks like the one below. (If nothing seems to happen when you click “Run Scan”, check the appendix for some troubleshooting ideas.)

Step 7: Let's verify that these stocks have an RSI(14) value that is above 80. Click on the name of the first stock in the list to see its chart. It should look similar to this:

(Add the RSI(14) indicator to your chart if it doesn't have it already.) Now check the RSI(14) value in the legend area of the chart. It should be greater than 80. (It is 80.08 in the screenshot above.) And that's it! In just these few quick steps we have found all of the stocks with an RSI value above 80. Of course, we need to do more research and possibly expand our search criteria to create a more manageable result set, but this demo shows how quickly you can get from looking for stocks to evaluating them.

Tutorial #2: Validating Scan Results With CandleGlance

In the previous tutorial, we validated just one stock. As we develop a new scan, it is a good idea to validate all of the results instead of just one or two. StockCharts' CandleGlance feature makes validating most scans a breeze.

Step 1: From the scan results page (Step 6 in our example above), choose “Store these results in a new ChartList” from the Available Actions menu. A dialog will pop up allowing you to name this new ChartList. We named this one “Scan Results”. This will create a ChartList where we will always put the temporary scan results. (Note: If there is already a ChartList called Scan Results, chose “Replace an existing ChartList with these results” from the Available Actions menu to overwrite the old contents.)

Once your results have been stored in a ChartList, you will see a new page with the results in what we call Summary view:

Step 2: Switch views to CandleGlance by changing 'Summary' to 'CandleGlance' in the drop down list at the top of the page. Mini-charts for all of the scan results will be displayed:

Step 3: Since we are doing a scan for RSI values, add RSI to the mini-charts. Do this by selecting RSI from the “Add Indicator” dropdown menu and clicking the green “Go” button. Now our mini-charts show RSI values below the main chart:

This three-step process is what you should get used to doing as you build your scan. By doing a visual inspection you can determine if the scan is giving the expected results. If not, adjust the scan logic as necessary. To continue getting familiar with the workbench, repeat these steps on your own for the scans shown below. We will look at stocks priced above $10 in order reduce the size or our result count:

MACD Cross:

[type is stock]
and [close > 10]
and [MACD Signal (12,26,9) x MACD Line (12,26,9)]
and [RSI (14) > 50]

Improving Momentum:

[type = stock]
and [close > 10]
and [CMF (20) > 0.2]
and [OBV > yesterday's OBV]

Again, take the time to create and run each of these scans using the Advanced Scan Workbench and then validate your results by first overwriting the contents of the “Scan Results” ChartList with your new results and then viewing the “Scan Results” ChartList in CandleGlance mode.

Now that we have run through a couple of basic scenarios to get familiar with the Scan Engine, let's go into more detail on how scans are created and what they consist of.

Scan Expressions

Scans in the Advanced Workbench are text based and consist of a series of items combined with 'and' or 'or' connectors. This entire string is collectively known as the 'scan expression' and is made of up from 'scan clauses'. Let's look at a scan commonly known as the Golden Cross. Here is the full scan expression:

[type = stock]
and [daily sma(20,daily volume) > 40000]
and [today's sma(50,close) > today's sma(200,close)]
and [yesterday's sma(50,close) <= yesterday's sma(200,close)]

These are the individual scan clauses that make up the scan expression:

  • [type = stock]
  • [daily sma(20,daily volume) > 40000]
  • [today's sma(50,close) > today's sma(200,close)]
  • [yesterday's sma(50,close) ⇐ yesterday's sma(200,close)]

Scan clauses are combined by using the AND and OR operators. Clauses are surrounded by brackets and evaluate to a true or false outcome. If the clause evaluates to true, the item (stock, mutual fund, index, etc) it is passed to the next clause to be further evaluated. If it is false the item is eliminated from further processing. Functions within the scan clause which take parameters are specified by using parenthesis and individual parameters are separated by commas. Here are some examples:

  • MACD Line(12,26,9)
  • Slow Stoch %D(14,3)
  • SMA(50,volume)
  • BB Width(20,2)

The following examples illustrate scan expressions made from scan clauses:

[type = fund] Shows all mutual funds (vs indexes, futures contracts, or stocks)
[type is stock] and [exchange = NYSE] Shows all items in the Scan Engine listed on the NYSE
[type = stock] and [exchange = NYSE] and [ Close > 30] Shows all NYSE stocks whose close is above 30
[open > 30] or [high > 30] Shows all symbols who either have an open or a high above 30
[Upper BB(20,2) < close] The close is above the upper Bollenger Band using parameters of 20 periods and 2 standard deviations

The 'OR' Issue

A common source of problems in scan expressions is when the OR operator is used to combine clauses. Unless your syntax is explicit, the results might not be what you intend. Here is an example:

A and B or C

Does this mean (A and B) or C? Or is it A and (B or C)? That slight difference can be a huge difference in results.

What will the following scan return?

[close> 20] and [country is US] or [country is canada]

Our intent may be to find all US or Canadian stocks above $20. In reality, however, this scan will find all US stocks above 20 and ANY Canadian stock.

The Scan Engine will treat everything prior to the 'OR' clause as a single entity. Thus:

(A and B) or C (A and B and C) or D

The Scan Engine is effectively doing the following (note extra brackets added):

[[close > 20] and [country is US]] or [country is canada]

what we want is this:

[close > 20] and [[country is US] or [country is canada]]

Finding this type of error can be very difficult, especially when the consequences are not as obvious this example. Misplaced or missing brackets can radically alter the results. If the scan uses an OR operator it is best to always add brackets around your clauses so you get what you expect.

We will spend a considerable amount of time building scan expressions later But before we start writing scans, we will explore a number of core skills, that when understood, will act as building blocks in creating more complex scans.

Universal Skills

There are certain common elements and patterns frequently found in scan expressions. The following is a list of core skills that will help when creating more complex scans:

Min/Max - Often it is important to know what the highest or lowest value for an indicator is over a time period. The most common example for the Min/Max functions is a 52-week high/low. You may also wish to know what the highest volume level was over the last week, or the lowest RSI value in the last month.

Crossovers - In technical analysis a trend change is often signaled by one line crossing over another. This can be a bullish signal or a bearish signal. Common crossovers consist of indicators such as moving averages, MACD signals or stochastics. A crossover is defined when indicator A is greater than indicator B today, but was less than indicator B yesterday.

% Ranges and Values - Another common task is to find out if a price or indicator value is within a specific range of values or is a multiple of another value. For example, we may want to know when a stock's price is within 5% of its 52-week high or volume is 25% higher than its average volume. For example multiplying a value by 1.05 will find values 5% greater. Multiply by .85 and we will find values 15% less (1.0 - .15).

Divergence - Divergences occur when one indicator is going in one direction and another indicator is going in the opposite direction. This is a potential sign that a change in trend may be about to take place. For example, if the price is going up, and the RSI doing down, we have a negative divergence. This would suggest the price may soon drop. A positive divergence is the opposite: price is falling and other indicators are rising.

Time offsets - The scan engine allows you to run a scan on any day that it has data for. If you wish to run your scan on dates in the past, you can use the date offset feature to tell the Scan Engine what date to look at.

These core skills will provide the basic knowledge to begin writing your own scans. We will continue by exploring the Advanced Scan Workbench features in more detail and will follow that by developing our own scans.

Using the Advanced Scan Workbench

Now that we have some basics under our belt, let's look at these areas of the user interface in a bit more detail. Here is the advanced scan workbench broken into functional areas:

Date Selection - The date selection area allows you to specify which day of data the scan is run against. The Scan Engine contains approximately four years of historic data. Using date offsets, you can run your scan on any given day the Scan Engine has data for.

In most cases that number will be 0 ,which implies we want to scan the latest date. You have further control by selecting if you wish that offset to be relative to the last closing date, or if the market is currently open, the latest intraday quote. During market hours the Scan Engine updates prices roughly every two minutes.

To run a scan on a previous date, click on the edit field containing the date offset. The date offset is the field immediately following the word 'Starting' in the Date Selection area. In this screen shot, the offset value is 0. Clicking in this field will bring up a calendar pop-up which will allow you to pick the date you wish to scan. The Scan Engine will only perform the scan on one date at a time. This is often referred to as the 'Within' limitation. That is to say, the Scan Engine cannot run a single scan over a date range. For example one may wish to run a scan for 52-week highs on the dates within the range of Sept 4th and Sept 7th. Currently this requires four separate scans using the appropriate date offsets.

Scan Management - This set of controls allows us to add, modify and delete saved scans. To save a new scan, click the “Save As” icon. A dialog box will appear and you can enter the name of the scan. If you are modifying an existing scan, the existing scan name is displayed. To run a saved scan, select it from the 'Your Favorite Scans' list and the Scan Expression editor will update with the saved scan. To remove a scan, first select it so that the editor is updated with the scan, then click the “Delete” icon and confirm its deletion.

Scan Component Lists - The Scan Component area allows you to construct a scan by choosing from lists of technical indicators, candlestick patterns, point & figure patterns and predefined scans. Additionally, the scope of the scan can be restricted to various indexes and sectors or industries. Certain fundamental data such as market capitalization, earnings per share and PE ratio are also available. To add a new scan clause, choose the item from the list that you are interested in and click the 'Add' button next to it. The scan expression editor will be updated with the default clause for that item. You are free to modify it as you like. For example, choosing 'Close' from the Price, Volume and SCTRs list will add the following:

[Close> 99.9]

The new clause will be appended to the existing scan expression using the 'AND' operator. You may edit the default values as required by your scan logic.

Comments - As the scan expressions become more complex, it can be harder to remember what effect various clauses have on the results. This can be especially true if you have not looked at the scan in some time. Look at the following scan. Do not worry about understanding what it does, just note how dense the text can get:

[type is stock] and [sma(20,volume)> 40000] and [country is US] and [Close> 15]
and [today's high> yesterday's daily max(260,high)]
and [today's high> yesterday's daily max(260,high)]
and [today's sma(50,close)> today's sma(200,close)]
and [yesterday's sma(50,close) <= yesterday's sma(200,close)]
and [yesterday's close <= yesterday's upper bb] and [close> upper bb]

We will make this easier to understand by using comments. A comment is a line of text in the editor that is not treated as part of the scan. There are two methods of adding comments. Any line which contains the ”#” character or two forward slashes ”/” will ignore all the text on a line up to the end of that line.

Here is the same scan with comments:

#
# My Test scan  -  January, 2012
#
# Stocks only
[type is stock]
# we need some healthy volume
and [sma(20,volume)> 40000]
# US stocks above $15
and [country is US]
and [Close> 15]
// 52 week high - we're now using double slashes for comments
and [today's high> yesterday's daily max(260,high)] // SMA 50 crosses over SMA 200
and [today's sma(50,close)> today's sma(200,close)]
and [yesterday's sma(50,close) <= yesterday's sma(200,close)]
// moves above upper Bollenger Band
and [yesterday's close <= yesterday's upper bb]
and [close> upper bb]

This version is much easier to understand. Anything that may be useful to note, such as the scan name, date, author or a description should be added as comments at the top of the scan. Another way to use comments is to create a single scan that can act as multiple scans by commenting out clauses. It is possible to create a single scan that does both 52-week highs and lows. The results depend on which line we uncomment:

#
# My 52 week dual scan  -  January, 2012
#
# Stocks only
[type is stock]
# we need some healthy volume
and [sma(20,volume)> 40000]
# US stocks above $15
and [country is US]
and [Close> 15]

//52 week high
and [today's high> yesterday's daily max(260,high)]

// 52 week low
// and [today's low <yesterday's daily max(260,low)]

This scan as written will find 52-week highs. To find 52-week lows, comment out the 52-week high clause and uncomment the 52-week low clause. And finally, a very important use of comments is in debugging scans. We will look at debugging later. First we will look at another debugging technique which is syntax validation.

Syntax Validation

The Scan Expression editor allows you to freely edit scan expressions. It also allows you to make mistakes. We all accidentally forget to add a closing bracket or miss a comma in separating parameters in a function. Syntax errors are generally typos and consist of missing or misplaced text. To help detect syntax errors, click the Check Syntax button. This will cause the Scan Engine to analyze the scan looking for mistakes. If errors are found, the area at the bottom of the Scan Expression editor will attempt to identify the problem so it can be corrected. It is a good habit to check the syntax frequently as you build the scan. Spotting mistakes after having added a single clause will be faster than trying to spot a mistake after having entered twenty. As we noted in the comments section above, putting each scan clause on its own line will allow you to more quickly identify the location of the problem. If the error is an incorrect keyword, select that keyword from the Scan Components area and click “Add”. This will add the default clause which is syntactically correct. Compare that clause against your clause to see what may be incorrect.

Thus far we have discussed an overview of the Scan Engine, the elements of the Advanced Scan Workbench, as well as terminology and concepts around scan expressions. We have also discussed best practices in writing scans and have taken steps to learn core skills in finding common patterns. Now we will put all these together to write a set of sample scans. We'll go through the process of developing a scan, improving its logic and debugging problems.

Building a Scan

Let's start by building a scan that will find US stocks whose 50-day moving average has crossed above the 200-day moving average on strong volume. The 50-day moving average crossing above the 200-day is generally seen as a bullish signal and is known as the Golden Cross. We will start with the default scan:

[type is stock] and [sma(20,volume) > 40000]

This will find all stocks whose 20-day simple moving average (sma) of volume is greater than 40,000 shares. This has the effect of selecting stocks that have some reasonable level of trading over the last 20 trading days.

Click the 'Run Scan' button to display the results. The Scan Engine will only display 999 results, which is too many results to be helpful. We must add more criteria to the scan to reduce the number of hits.

As we add more clauses to the scans our result set should steadily decrease to a reasonable number that can be further evaluated. What is 'reasonable' will vary from person to person, but it is a good idea to run the scan as you add clauses to ensure you haven't made the scan so specific that you get 0 results, or conversely, that you still have hundreds of results.

First, we will restrict the list to US stocks while using the good practice of employing comments. Choose the Country option from the Ticker Properties list and click Insert. Note that when we insert a new clause it is appended to the existing one. We need to manually add carriage returns and comments.

We now have:

// Find stocks vs mutual funds, indexes or futures
[type is stock]
// Trades at least 40K Shares daily this last month
and [sma(20,volume) > 40000]
// US Stocks only for now
and [country is US]

Next we'll add the moving averages (MA). The general format for a simple moving average (SMA) looks like:

sma(50,close)

where the first parameter is the time period and the second is the value being averaged. The value can be a number of different things, but is often volume or the closing price. In our case, it will be the closing price.

The 50-day MA crossing the 200-day MA can be expressed with the following clauses supplied by choosing the 'Bullish 50/200-day MA cross' option from the Predefined Scans list in the Scan Components area:

[today's sma(50,close) > today's sma(200,close)] and [yesterday's sma(50,close) <= yesterday's sma(200,close)]

This says that today the 50-day MA is above the 200-day and yesterday the 50-day MA was below the 200. This implies that today the 50-day MA has crossed above the 200-day MA.

Now our scan expression looks like this:

// Find stocks vs mutual funds, indexes or futures
[type is stock]
// Trades at least 40K Shares daily this last month
and [sma(20,volume) > 40000]
// US Stocks only for now
and [country is US]
// 50 day SMA crosses above 200 SMA
and [today's sma(50,close) > today's sma(200,close)] and [yesterday's sma(50,close) <= yesterday's sma(200,close)]

Running this scan gives a much smaller list, so we're making progress. Remember to check the results as more clauses are added by saving them to a ChartList and using CandleGlance to verify them.

In technical analysis, one line crossing another is often a meaningful signal. Our initial criteria contained two clauses to find this crossover event, but there is a shortcut. The “crosses” operator will do this in only one clause. The crosses operator is signified by the 'x':

[today's sma(50,close) x today's sma(200,close)]

Using this new method we have simplified our scan, but get the same results:

// Find stocks vs mutual funds, indexes or futures
[type is stock]
// Trades at least 40K Shares daily this last month
and [sma(20,volume) > 40000]
// US Stocks only for now
and [country is US]
// 50 day SMA crosses above 200 SMA
and [today's sma(50,close) x today's sma(200,close)]

The “crosses” operator might be more accurately called the “crosses above” operator. It will find items where the first term (50 day SMA) crosses above the second term (200 day SMA). A common question is: “Where is the crosses below function?”. To achieve the crosses below functionality, merely switch the order when using the crosses operator. When line A crosses below B, by definition B has crossed above A.

The final task in this scan is to find these bullish crossovers that have happened with strong volume. To do this we need to compare today's volume with a value for previous volume. First we can create a clause to see if today's volume is greater than yesterday's:

[volume > yesterday's volume]

This is OK, but this could be true if today's volume is greater than yesterday's volume by only one share. We want strong volume. How about:

[volume > yesterday's volume * 1.5]

This will find stocks whose volume today is more than 150% of yesterday's volume. Now we have:

// Find stocks vs mutual funds, indexes or futures
[type is stock]
// Trades at least 40K Shares daily this last month
and [sma(20,volume) > 40000]
// US Stocks only for now
and [country is US]
// 50 day SMA crosses above 200 SMA
and [today's sma(50,close) x today's sma(200,close)]
// Volume up 150% from yesterday
and [volume > yesterday's volume * 1.5]
// Find stocks vs mutual funds, indexes or futures
[type is stock]
// Trades at least 40K Shares daily this last month
and [sma(20,volume) > 40000]
// US Stocks only for now
and [country is US]
// 50 day SMA crosses above 200 SMA
and [today's sma(50,close) x today's sma(200,close)]

// Volume up 150% from 50-day SMA
and [volume> sma(50,volume) * 1.5]

And make sure today's price was above yesterday's as we want stocks that are going up:

// Find stocks vs mutual funds, indexes or futures
[type is stock]
// Trades at least 40K Shares daily this last month
and [sma(20,volume) > 40000]
// US Stocks only for now
and [country is US]
// 50 day SMA crosses above 200 SMA
and [today's sma(50,close) x today's sma(200,close)]
// Volume up 150% from 50-day SMA
and [volume > sma(50,volume) * 1.5]
// Price goes up!
and [close > yesterday's close]

At this point, we've gone from thousands of stocks to only a handful which we can further evaluate. Be aware that on any given day a scan that is more specific like this one may not return any results depending on what the market is doing.

For the purposes of this tutorial, if you are getting no results, you can try using the date offset to have your scan run on days in the recent past or changing other scan parameters. You can also quickly modify this example to look for stocks whose 50-day SMA crosses below the 200-day SMA. Recall the discussion on how to change the direction of the crosses operator.

This scan exhibits two of our Universal Skills: the crossover and using percentages of values.

Divergences

The next Universal Skill we will look at is divergences. Divergences happen when one metric is rising and another is falling (or vice versa). Typically the first metric is price and the second may be a number of different technical indicators. A divergence can signal an upcoming change in trend.

For this scan, we'll first need to find a method of spotting a trend. The Advanced Scan Workbench has a number of methods that can be used to spot trends. One of the simplest is checking that the price has risen (or fallen) by a certain percentage over a certain time window:

[close > 10 days ago close * 1.05]

This simply looks for a five percent increase over the price ten days ago. Note that we're comparing just two price points, today and ten days ago. The price could move wildly between those two dates and still satisfy our criteria. A more readable way of expressing this is using the PctChange function:

[PctChange(10,close) > 5]

An alternate method is to check that each day the price is greater than the previous:

[close> yesterday's close ]
and [yesterday's close > 2 days ago close]
and [2 days ago close > 3 days ago close]
etc..

This version ensures that every day the close is higher than the previous close, but does not specify by how much. Yet another is to use the Candlestick Building Blocks. The Candlestick Building Blocks allow you to create scans from common Candlestick patterns. In this case, we'll use the UpTrend pattern:

[Uptrend = true]

The Uptrend pattern will find stocks whose current price is above the midpoint of the current daily range and is above the 10-day Exponential Moving Average (EMA). Note that the UpTrend pattern can only be used for prices.

And finally, we can also use the max and min functions:

[close > yesterday's max(10,close)]

which says today's close is greater than the highest closing price over the previous ten days. All of these can be used to spot a trend, but none will necessarily be perfect. You should experiment with different methods to see which works best for you.

Some of the more popular indicators used for divergences are the Chaikin Money Flow (CMF), On Balance Volume (OBV), Moving Average Convergence/Divergence (MACD) and the Relative Strength Indicator (RSI). We'll use the CMF to see what stocks are making higher highs, but the money flow is going down.

// US Stocks over $10 with some healthy volume
[type = stock]
and [country = us]
and [daily sma(20,daily volume) > 40000]
and [close> 10]
// Price up 10%+ over past 20 days
and [PctChange(20,close) > 10]
// Chaikin Money Flow falling
and [CMF(10) < yesterday's min(10,CMF(10))]

Another variation might use On Balance Volume (OBV) instead:

// US Stocks over $10 with some healthy volume
[type = stock]
and [country = us]
and [daily sma(20,daily volume) > 40000]
and [close> 10]
// Price up 10%+ over past 20 days
and [PctChange(20,close) > 10]
// On Balance Volume falling
and [OBV < yesterday's min(10,OBV)]

No scan is perfect of course. Evaluate the results that the Scan Engine returns and modify the scan to narrow in on a set of criteria that gives you the best set of results. Try different methods for spotting trends, different date ranges and percentage changes.

Strong Sectors

As the saying goes, “There's always a bull market somewhere”. Frequently one part of the market is doing well while others are not. In times of trouble groups like Healthcare and Utilities do well while Technology is often leading the market out of a downturn. Being able to focus on sectors and industries will help find good candidates at any time.

We'll start again with our basic scan that looks for stocks with a healthy price and volume:

// Sector Scan Example
[type is stock]
and [daily sma(20,daily volume) > 40000]
and [close > 10]

Use the Sectors and Industries list in the Scan Builder section to add the following sectors: Consumer Discretionary, Technology and Industrials. Recall the discussion about the OR operator and update the scan expression as follows:

// Sector Scan Example
[type is stock]
and [daily sma(20,daily volume) > 40000]
and [close > 10]
// Just these sectors only
and [[group is TechnologySector] or [group is IndustrialsSector] or [group is ConsDiscretionarySector]]

Find those that are showing strength and are within 5% of their 52-week high:

// Sector Scan Example
[type is stock]
and [daily sma(20,daily volume) > 40000]
and [close> 10]
// Just these sectors only
and [[group is TechnologySector] or [group is IndustrialsSector] or [group is ConsDiscretionarySector]]
// How many are near 5% of their 52 week high?
and [today's high > yesterday's daily max(260,high) * .95]

and finally find those with volume that is at least 25% higher than the 50-day simple moving average:

// Sector Scan Example
[type is stock]
and [daily sma(20,daily volume) > 40000]
and [close > 10]
// Just these sectors only
and [[group is TechnologySector] or [group is IndustrialsSector] or [group is ConsDiscretionarySector]]
// How many are near 5% of their 52 week high?
and [today's high > yesterday's daily max(260,high) * .95]
// Better than average volume
and [volume > sma(50,volume) * 1.25]

This scan can be adapted to monitor any sector or industry to find the strongest stocks in the hottest areas.

Bullish Percent Indexes

Our final scan will create our own Bullish Percent Index using Point and Figure clauses. A Bullish Percent Index (a.k.a. BPI) is simply a calculation of how many stocks in a group have Point and Figure Buy Signals, expressed as a percentage. The group can be anything, but is typically a common index like the S&P 500 or a sector such as Healthcare or Transportation. If the Transportation BPI is above 90, that means that 90% of the stocks in this group currently exhibit a Point & Figure Buy Signal. This gives us a gauge of the overall bullishness of that particular group.

A Point and Figure Buy signal occurs when a column of Xs exceeds a prior column of Xs. For more information on Point and Figure charts, visit ChartSchool. We will start with a simple example using the Nasdaq 100 (100 is easy to calculate a percentage from):

// P&F Buy Signal
[Buy Signal is true]
// Nasdaq 100
and [group is Nasdaq100]

Take the number of results and divide by 100 and we have a BPI value for the Nadsaq 100. As it turns out, this is a common BPI and is automatically calculated by StockCharts under the symbol $BPNDX. We can take this scan and extended it to any grouping we wish. We merely need to know the total number of members in that group with a buy signal divided by the total number of members in the group.

A reasonable group size is approximately 20 stocks or more. Anything less will lead to larger swings in the BPI value which may prove less useful. Next, let's try a defined group, but one that is not automatically calculated. From the Indices and ETFs list, choose the PHLX Gold Mining Index:

// Philadelphia Gold Mining Index
[group is XAU]

First run the scan to see how many stocks are in the group. As of this writing, there are 17. Now add our Buy Signal filter:

// Philadelphia Gold Mining Index
[group is XAU]
// Buy Signal
and [Buy Signal is true]

On this day there are 12 with a Buy Signal, thus our BPI for this group is 12/17 = 70.6%. Now let's try one with our own group.

Go to the ChartLists dropdown menu, choose one of your ChartLists and click “Add”. Perhaps you've saved a group of stocks from the Sector and Industries list or you have created a ChartList by hand. In this example assume there is a ChartList consisting of International ETFs (Exchange Traded Funds). The scan expression will now have a clause similar to:

// My group of International ETFs
[favorites list is 9]
// Buy Signal
and [Buy Signal is true]

Note that the Scan Engine numbers ChartLists vs using the list's name. This scan may give us a clue how well International markets are behaving. We run this scan twice. Once with the Buy Signal clause commented out to get the total number, then again to get the number showing Buy Signals. We then divide the former by the latter to get this group's BPI.

We can then add other parameters such as the Point and Figure Triple Top Breakout to find those who not only have a buy signal, but have broken above two prior highs:

// My group of International ETFs
[favorites list is 9]
// Buy Signal
and [Buy Signal is true]
// Triple Top Breakout
and [Triple Top Breakout = true]

In this section we've developed a number of scans representing all of the universal skills we talked about earlier. Feel free at this point to play with any of these scans (did you save them?) to see how they behave as you adjust various parameters, or add new ones. The best way to learn is to experiment!

Next, we will look at one of the necessary skills you will have to acquire if you intend to do a lot of scanning. Writing scans does not always go as hoped, and the more complex the scan, the more likely something unintended will occur. We will need strategies to debug and fix these problems.

Debugging

As much as we might try to create the perfect scan we often have problems. Consider the following scan:

[type = stock]
and [daily sma(20,daily volume) > 40000]
and [Upper BB (20,2) < close]
and [RSI (14) > 50.0]
and [%B (20,2) < 0]
and [close > 12]

This scan will never return anything. How do we begin to solve this problem? In general, the best way to debug is to break the scan down to something very basic and add clauses one at a time while checking the results at every step. Repeat this process until the results are not what you expect.

Start by commenting out clauses to get back to something that works as expected:

[type = stock]
//and [daily sma(20,daily volume) > 40000]
//and [Upper BB (20,2) < close]
//and [RSI (14) > 50.0]
//and [%B (20,2) < 0]
//and [close > 12]

We've basically turned the clauses into comments that the Scan Engine ignores. This scan now shows us all stocks ([type is stock]) up to 999 results. Uncomment the next line:

[type = stock]
and [daily sma(20,daily volume)> 40000]
//and [Upper BB (20,2) < close]
//and [RSI (14) > 50.0]
//and [%B (20,2) < 0]
//and [close > 12]

We're still getting lots of results, so keep going, removing each line's comment and checking the results.

Eventually, we will find that uncommenting [%B (20,2) < 0] will cause the results to go to 0. Why? It has to do with the Upper BB clause we saw earlier. The [Upper BB (20,2) < close] clause says that the closing price is above the upper Bollinger Band. However, [%B (20,2) < 0] is true when the price is below the lower Bollinger Band. It is impossible to satisfy both of these as it would require the price to be both above the upper and below the lower band at the same time.

Making this change will fix the problem:

[%B (20,2) < 0]
to
[%B (20,2) > 0]

A simple error in the operator will cause this scan to return 0 results. An error of this sort can take quite a bit of time to discover, especially if you have a scan with dozens of complex clauses. Once you have identified the offending clause you can determine if there is a bug in the clause, or if the clause is correct, but is too selective.

Sometimes when trying to debug a scan we get far too many hits as we broaden our scope to find the problem. We can reduce this by using the 'Starts with' trick. Adding the following clause will ensure only symbols that start with 'A' are displayed (note that the search is not case-sensitive):

[symbol starts with 'A']

This can be a handy way to reduce clutter in your test results. Further, if you are looking to see if a specific symbol is present you can expand on this:

[symbol starts with 'AAPL']

This will only show Apple in the results page. In some cases looking at a single stock and its chart can help us discover errors in the scan logic. It can also help confirm the logic. If you expect a specific stock to be in the results, you can narrow the focus to just include that stock. Then proceed to add clauses and notice when the results change. This is easier than having to look at a large list of results to see if the stock is there.

Debugging scans in the past

When debugging scans using date offsets it is helpful to create a chart that coincides with the date of the scan results. This can be done by using the Select Start/End option and using the calendar popup to choose the appropriate date. Adding the indicators you are interested in to the chart will display those values as they existed on the requested date.

Note that the values in the charting engine are typically rounded to two or three decimal points where the Scan Engine is sensitive to values down to six decimal places.

Help!

Scanning is as much art as it is science and it will take practice to get comfortable with it. The best way to learn is to experiment as much as possible. You can find many ways to accomplish the same general task with slightly different results. However, if you are having problems, here is a list of what you should try:

Break the scan down to something as simple as possible. Add clauses one at a time, checking both your syntax and results as you go. Use comments to turn on and off clauses. Is the additional clause giving you the results you're looking for? If not, you may need to adjust it before proceeding.

Samples are an excellent way to learn how to express your thoughts as scan clauses. In the Scan Components area is a list of Predefined Scans. These popular scans will demonstrate how to do a number of common types of scans such as finding rising or falling indicators, crossovers and various buy and sell signals. Take some time to choose each one of these in turn and see what logic the Scan Engine is using to implement these scans.

Visit the Scanning Technically blog. Here you will find announcements about new Scan Engine features as well as tips on how to perform certain types of scans.

Search the StockCharts Answer Network (known as S.C.A.N.). Many fellow scan users have asked questions and have gotten answers which show different strategies for common problems. Seeing how someone else went about creating a scan and other users' suggestions is one of the best ways to learn new methods of finding stocks.

If you still are having problems you can contact the support team. While the support team can't necessarily write your scan, they can assist if you can't get past a problem. When sending in a support request, provide the following information:

  • What is your scan expression?
  • Is the date offset something other than 0?
  • Is this an intraday scan or end of day scan?
  • Are you getting syntax errors, or results you don't feel are valid?
  • Give specific examples. i.e. I ran my scan on Jan 10th using intraday data and got back IBM as one of the results. Looking at the chart, I don't understand why it is there.

The faster the support team can understand exactly what you have tried and what the problem is, the faster you are likely to get a resolution.

Conclusion

The Scan Engine is one of the most powerful tools available at StockCharts. Learning the nuances of scanning as it relates to your investment objectives does take a bit of time. However, the time you will save in the long run by finding more winners quickly will vastly outweigh the time spent learning. The key to becoming an expert scanner is to continue to experiment with different ways of finding your best leads.

Additional Resources

For more information and samples of scans using the Advanced Scan Workbench, see the Advanced Scan Library

This page contains links to numerous sample scans, the Scan Engine's predefined scans, scans sourced from books and articles as well as scans contributed by other StockCharts users. Reviewing the Sample Scans is a worthwhile effort for all levels of scanning abilities.