SAS Base Programming study materials
SAS Base Programming study resources

PROC IMPORT guessed the types from the first 20 rows

SAS Base Programming · Put your knowledge to work in context.

All case studies

PROC IMPORT guessed the types from the first 20 rows

A vendor sends a 60,000-row pipe-delimited file with a header row. The developer writes:

proc import datafile='stores.txt' out=work.stores dbms=dlm replace; delimiter='|'; getnames=yes; run;

GUESSINGROWS is left at its default. store_id comes in numeric, so '00123' is stored as 123. region is blank in the first 20 rows and is created as character length 1, so 'Northeast' appears as 'N'. sqft is numeric, but the 40 rows where the vendor wrote '1,200' are missing. The load reports no error and the row count is exactly 60,000. Downstream, a match to the store master on store_id fails for every store, and a regional roll-up shows 58,000 rows in a group called 'N'.

1. Explain each of the three defects in terms of how PROC IMPORT decides types and lengths.

Read the worked answer

PROC IMPORT does not read the file's structure; it samples it. With GUESSINGROWS at the default of 20, it reads the header plus the first 20 data rows (using SCANOVER and a 32K buffer, but still 20 rows), infers a type and a length for each column, generates a DATA step, and runs it over the whole file.

  • store_id -> numeric. Every sampled value looked numeric, so the column was typed numeric. Numeric storage has no concept of leading zeros, so '00123' and '123' become the same number; a character join key downstream can never match. Nothing is lost - the information was simply never guaranteed to exist as text.
  • region -> length 1. All 20 sampled values were blank, so the widest observed value had width 0 and SAS gave it a minimum length of 1. The subsequent generated INPUT uses a $1. informat, so 'Northeast' is truncated to 'N' - silently, because truncation on input is not an error. The roll-up group 'N' with 58,000 rows is the tail of that.
  • sqft with '1,200'. The 40 offending rows are past row 20, so the numeric guess was made without seeing them. The generated statement does not use the COMMA informat, so the embedded comma triggers an invalid-data condition for those rows; with the generated step's modifiers the value lands as missing rather than stopping the load.

The 60,000-row count proves none of this: row count only shows that records were read, not that fields were preserved.

2. Write the replacement step and state why it is the correct choice for a production load.

Read the worked answer
data work.stores;
  infile 'stores.txt' dlm='|' dsd truncover lrecl=32767 firstobs=2;
  length store_id $5 region $20 sqft 8;
  input store_id :$char5.
        region   :$char20.
        sqft     :comma9.
        ;
run;
  • An explicit length statement plus $char informat keeps leading zeros and never truncates.
  • comma9. handles '1,200' correctly, so those 40 rows carry a real value instead of a missing one - and a comma in a field is legitimate here because the delimiter is |, so dsd quoting protects it.
  • firstobs=2 skips the header, truncover prevents a short line from pulling the next record forward, and lrecl stops a long line being split.

The judgement call is that PROC IMPORT is an exploration tool. It re-guesses the layout on every run, so any vendor change - a new spelling, a blank block, a comma - silently changes types and lengths, and the generated step is not something you reviewed. GUESSINGROWS=MAX narrows the window but still guesses; a hand-written INPUT with explicit informats encodes the contract. If the file layout genuinely varies, read everything as $char and convert with the INPUT function afterwards so the conversions are visible and testable.

3. What would you check before letting this dataset feed the store-master match?

Read the worked answer

Checks that would have caught all three defects:

proc contents data=work.stores; run;                 /* type and length, not just names */
proc sql;
  select count(*) as rows,
         count(distinct store_id) as stores,
         sum(store_id is missing) as miss_id,
         sum(sqft is missing) as miss_sqft,
         max(length(region)) as max_region_len
  from work.stores;
quit;
proc freq data=work.stores; tables region / missing; run;

Specifically: store_id must be character (PROC CONTENTS type), row count must equal the vendor's control total of 60,000, max_region_len should be the observed maximum and not 1, missing counts on sqft should be explained rather than assumed, and the region frequency list should contain no single-character values. Then a reconciliation against the store master - how many of the 60,000 keys match - is the final gate, because that is the failure the requester actually feels.