NB_430_22_4 - NB 430-22-4 SOI – Fiscal Year (FY) 2023 Annual Soil Refresh Requirements
NB 430-22-4 SOI – Fiscal Year (FY) 2023 Annual Soil Refresh Requirements
National Bulletin: 430-22-4 Date: June 16, 2022
Subject: SOI – Fiscal Year (FY) 2023 Annual Soil Refresh Requirements


                                                     Regional Office Action Required By:  August 19, 2022

                                             State Soil Scientists Action Required By:  September 16, 2022
 
Purpose.  To provide requirements on the annual refresh of the official soil survey data.
 
Expiration Date.  September 30, 2023
 
Background.  Each year, the official soil survey data is updated. The official soils data is accessed by the web-based Web Soil Survey (WSS) or is queried through the Soil Data Access (SDA) web service.

The users of this data rely on its accuracy as they make decisions concerning land management, including Government officials responsible for implementing Government farm and other programs established by law. This refresh of the official soil dataset occurs annually with the data being delivered by October 1.

A very detailed process of quality control (QC) and quality assurance (QA) is performed annually to ensure that the most up-to-date soils information is delivered. The soil survey regional offices are responsible for completing a QA review of new or revised data posted to the staging server. The State soil scientist (SSS), or their designee, is responsible for completing the QC review and exporting the certified soil survey data for their States to the staging server. The SSS is responsible for verifying and committing the certified spatial and attribute data from the staging server to the Soil Data Warehouse (SDW).

The FY 2023 annual soil refresh is the first time Raster Soil Surveys (RSSs) are being included. Even though the RSSs are not yet archived in SDW or posted to WSS, it is important to refresh the data on an annual cycle. This is particularly important to ensure that the RSSs contain the most up-to-date interpretations.

The goal for this year’s annual refresh is to complete the update of the soil database by October 1, 2022.

Deadlines:

      Soil Survey Geographic Database (SSURGO) and RSS

  1. July 29th – Soil Survey Offices cease all data entries into the National Soil Information System (NASIS) and complete any projects they need uploaded for the next refresh. Tabular and spatial edits must be completed.
  2. July 29th – Soil Services and Information completes all interpretations for the FY 2023 annual refresh.
  3. August 19th – Regional offices complete all QA and correlations.
  4. August 26th – Regional GIS Specialists post spatial data to the staging server.*
  5. September 16th – SSSs complete their review, and soil survey areas (SSAs) are exported and committed from the staging server to the warehouse.*
  6. September 23rd – National Soil Survey Center (NSSC) completes the cleanup, checks on all warehouse changes, and moves SSURGO to Soil Data Mart offline.
  7. September 30th – Contractors in Ft. Collins complete their checks and “flip the switch” to release the latest soil data updates to the public.

*In FY 2023, the spatial and tabular RSS data is NOT posted to the staging server and is NOT committed to the SDW. Please contact the Soils Hotline at (402) 437-5379 if you have additional questions.
 
Explanation.  This year’s annual refresh will require all surveys to be processed. The tabular data for every published SSA must be exported from NASIS to the staging server. This requirement includes the NOTCOM (“not complete”) SSAs. The spatial data is only posted for SSAs that had edits to the spatial data.

Update FY 2023 Surveys

Tabular data for surveys that have been updated and are ready to be published to WSS must be exported from NASIS on or after August 20, 2022, and processed as outlined below in steps 1-4.

Soil survey information published to the WSS will include the following:

  1. All required national interpretations are automatically included when a survey is exported from NASIS to the staging server. Required national interpretations include all NASIS primary rules marked as ready to use in the “NSSC Pangaea Rules” folder. The SSS can supplement the national interpretations by selecting interpretations of interest from the “NSSC Regional Interpretations” folder or interpretations tagged for State use from any of the other “Regional Office” folders. These are located under the “select/view interps” export tab in NASIS. Any SSS who has questions about national interpretations or NSSC-owned regional interpretations should contact Bob Dobos by email at bob.dobos@usda.gov. Any SSS who has questions about regional interpretations (non-NSSC owned) should contact the regional office.
  2. At a minimum, the component text, as listed in the column named “Group,” and nontechnical description, as listed in the column named “Kind,” must be checked for export under the “select/view text” export tab in NASIS. Verification, by the SSS, that all nontechnical descriptions are up-to-date and available for export can be determined by running the NASIS NSSC Pangaea Report named “NASIS to Staging Server Export - Brief Soil Descrip Check.” A web-based version of this report is available here. The web report may time out for larger States.
  3. Any other text table data considered for export to WSS must be examined by the SSS to verify that personally identifiable information (PII) is removed or not published.
  4. It is recommended that all components be exported, but the discretion of the SSS should be applied to the decision of whether minor components should be exported. At a minimum, every minor component that has been classified as a hydric soil must be exported.

The SSS will review and verify the data in the county and major land resource area overlap tables. The SSS will finalize documentation of the legend certification history table before export.

Schedule of Deadlines

To meet the October 1 deadline, a recommended schedule of sub-deadlines is as follows:

  • Regional offices should complete their work by August 19th.
  • The SSS should complete his or her work by September 16th.
  • The NSSC and Fort Collins staff should finish publishing the refreshed data to WSS starting by October 1st.

SSURGO Refresh Pre-Check Reports

When a survey is exported from NASIS to the staging server, a series of validations are run in a hidden structured query language (SQL) database stored procedure called “Check Fatal Errors.” If a soil survey fails any of these validations, the export to the staging server will not be successful. This stored procedure was updated in May 2022 and the following checks were added:

  1. Legends that have either a NULL geographic applicability or a geographic applicability not equal to "current wherever mapped."
  2. Legends that use a unique datamapunit as representative more than once in the correlation table.
  3. Mapunits that do have a single datamapunit flagged as representative in the correlation table. Look for instances of more than one representative datamapunit or no representative datamapunit.
  4. Datamapunits that have a sum of relative value (RV) component percentages greater than 100%.
  5. Datamapunits that do not have any component records.
  6. Components that are flagged as major but have an RV component percent less than 10 or have a NULL RV component percent.
  7. Components that have a duplication of both the horizon top and bottom RV depth. For example, a soil with an A horizon depth of 0 - 10 cm and Bt horizon depth of 0 - 10 cm would be an error.
  8. Components that have a NULL horizon RV top or bottom depth.
  9. Components with horizon top and bottom RV depths that have gaps or overlaps.

Click here for the comprehensive list of all validations included in the stored procedure.

The report is located in the “NSSC_Pangaea Report” folder and is named “NASIS to Staging Server Export - Fatal Errors Pre Check.” A web-based version of the report is also available here. When an error is found, the report calls a sub-report. The sub-report identifies where the errors are occurring and directs the SSS to those errors. This should greatly increase the efficiency of the SSS review by removing the time-consuming process of searching the errors and lighten the load on the server when the data is committed.

Over 3,200 surveys are exported each year. The servers that process exports have a finite capacity and the process slows when numerous surveys are exported at once. Delays are common close to the final deadline when export traffic increases. To avoid delays, it is strongly recommended that SSSs do not wait until the last minute to complete exports.

Additional information that is specific to the export process is in chapter 17 of the NASIS User Guide. Is also found in YouTube NASIS Training Videos.

The WSS refresh job aids and videos are located at the following URLs:

Export and Commit Official Soil Survey Data: https://www.nrcs.usda.gov/wps/portal/nrcs/detail/soils/edu/ncss/?cid=nrcs142p2_054322#database_gen

Recall* and Commit Official Soil Survey Data: https://www.nrcs.usda.gov/wps/portal/nrcs/detail/soils/edu/ncss/?cid=nrcs142p2_054322#database_gen

*The preferred method is to export from NASIS. Recall should only be used when absolutely necessary.

Guidance on the official soil survey information refresh policy has been written into Title 430, National Soil Survey Handbook (NSSH), Part 644, Subpart A, Section 644.2. Future guidance on the refresh of soil survey data will be referred to NSSH policy.
 
Contact.  For further information, please contact Drew Kinney, National Leader for Soil Business Systems, NSSC, at (402) 437-4098 or drew.kinney@usda.gov.    
 
 

 /s/

LUIS TUPAS
Deputy Chief, Soil Science and Resource Assessment

 
 

[NB_430_22_4 - ]