Friday, May 27, 2022

Reliability of Occupational Requirements Survey

 Some vocational witnesses and ALJs have taken to the air, talking about something that they don't really know anything about.  The Occupational Requirements Survey is not done and is therefore not reliable.  The ORS publishes the final first wave data (2018) and the preliminary second wave data (2021).  

The Bureau of Labor Statistics publishes its Handbook of Methods for the ORS.  The Handbook of Methods describes is calculations.  The chapter on calculations contains a subchapter, Reliability of ORS estimates.  In that subchapter, BLS states:

To assist users in confirming the reliability of ORS estimates, the ORS publishes standard errors. Standard errors provide users with a measure of the precision of an estimate to ensure that it is within an acceptable range for their intended purpose. The standard errors are calculated from collected and imputed data. The ORS program is researching methods for estimating the variance excluding imputed values. Examples on how to build confidence intervals using standard errors are included in the standard error section of the ORS website.

 At the bottom of the subchapter, BLS states:

Before publishing any estimate, the ORS program reviews it to make sure that it meets specified statistical reliability and confidentiality requirements. See data review and estimation tabs on the research section of the ORS website. Estimates that are consistent with these procedures are designated as fit for use and released in BLS publications.

Following the research section hyperlink, we get to the statement of estimation:

 Follow the imputation of methodology.  In 2016, the authors speaking for BLS state:

The survey plans to publish all estimates that meet the reliability and confidentiality criteria.

 The collection surveys have OMB clearance #1220-0189.  The design evaluation studied the methodology to "accurately and reliably capture data that are relevant to the SSA's disability program."  The Dynamics of Occupational Change: Implications of the Occupational Requirements Survey states:

While exclusive reliance on adjusting occupational employment weights is not a realistic prospect for ORS to provide up-todate information on job skill requirements, SSA has been unable to incorporate any occupational employment weights in its determinations of available work because of the nature of the DOT. Because the ORS uses standard occupational codes, linking occupation weights and skill scores will not be a problem, and the ORS will capture effects of changes in occupational size over time for SSA’s eligibility determination process for the first time. 

Does the ORS have limitations and problems?  Probably but that begs the economics question:

Compared to what?

Challenge the vocational witness about his/her methodology - compare to the BLS.  Challenge the vocational witness's ability to accumulate and synthesize data -- compare to the BLS.  How does not knowledge about work in nation and the ability to collect and report data about work in the nation compare to the BLS?  


Suggested Citation:

Lawrence Rohlfing, Reliability of Occupational Requirements Survey, California Social Security Attorney (May 27, 2022) 

Thursday, May 19, 2022

Storage-Facility Rental Clerk -- Using County Business Patterns to Crush the JBP Job Numbers Estimate

 Storage-facility rental clerk is often identified by vocational witnesses in Social Security disability hearings.  It is reasoning level 3, mathematics level 3, and language level 3.  It requires frequent reaching, handling, and fingering.  Job Browser Pro ver. 1.7.4 estimates 68,736 jobs, almost all of them in the real estate industry subsector (531000).  The question turns on reliability.  

Real estate fits with the general idea of a storage-facility rental clerk.  The Occupational Employment and Wage Statistics breaks down counter and rental clerk employment in the real estate and rental and leasing sector (NAICS 530000) into two subsectors: real estate (NAICS 531000) and rental and leasing services (NAICS 532000). (This is a free report in, just register, no subscription needed).  Rental and leasing services are the industries that employ furniture-rental consultants and automobile rental clerks.  Real estate still fits but seems broad, so we drill down.  

County Business Patterns reports employment for the real estate subsector at 1.7 million jobs in all occupations.  I run the CBP report for 3 or 4 digit NAICS code, starting with NAICS 531 in  (This report is subscription required.  CBP is a bear to work through in its native environment, last time I checked).  I get the results for the subsector:

NAICS        Description                                                                                               Jobs


Real estate



Lessors of real estate



Lessors of residential buildings and dwellings



Lessors of residential buildings and dwellings



Lessors of nonresidential buildings (except miniwarehouses)



Lessors of nonresidential buildings (except miniwarehouses)



Lessors of miniwarehouses and self-storage units



Lessors of miniwarehouses and self-storage units



Lessors of other real estate property



Lessors of other real estate property



Offices of real estate agents and brokers



Offices of real estate agents and brokers



Offices of real estate agents and brokers



Activities related to real estate



Real estate property managers



Residential property managers



Nonresidential property managers



Offices of real estate appraisers



Offices of real estate appraisers



Other activities related to real estate



Other activities related to real estate


The industry group for lessors of real estate contains specific industries for buildings and dwelling as well as nonresidential buildings except miniwarehouses.  Storage-facility rental clerk works in the minwarehouses and self-storage units business.  We check back with the DOT to be sure:

295.367-026 STORAGE-FACILITY RENTAL CLERK (business ser.; retail trade) Leases storage space to customers of rental storage facility: Informs customers of space availability, rental regulations, and rates. Assists customers in selection of storage unit size according to articles or material to be stored. Records terms of rental on rental agreement form and assists customer in completing form. Photographs completed form and customer to establish identification record, using security camera. Computes rental fee and collects payment. Maintains rental status record and waiting list for storage units. Notifies customers when rental term is about to expire or rent is overdue. Inspects storage area periodically to ensure storage units are locked. Observes individuals entering storage area to prevent access to or tampering with storage units by unauthorized persons. Loads film into security and surveillance cameras, records dates of film changes, and monitors camera operations to ensure performance as required. Cleans facility and maintains premises in orderly condition.

GOE: 09.04.02 STRENGTH: L GED: R3 M3 L3 SVP: 2 DLU: 86

The lead statement "summarizes the entire occupation. It offers essential information."  DICOT, introduction.  The storage-facility rental clerk works in storage facility businesses.  NAICS 531000 is too broad.  Storage-facility rental clerk has an apparent conflict with every industry in this list except lessors of miniwarehouses and self-storage units (NAICS 531130).  This specific industry accounts for 47,386 jobs in all occupations, not just the counter and rental clerks.  

This observation means that 68,736 jobs in the industry subsector has a conflict with the DOT once the claimants asks for administrative notice of CBP.  Opening up the Occupational Outlook Handbook for counter and rental clerks in (another free report), I follow the hyperlink to the Employment Projections for SOC 41-2021.  Within the real estate subsector, counter and rental clerks make up 30.4% of the jobs.  Instead of 68,000 jobs, the gross number of full- and part-time jobs is about 16,000 jobs.  Only full-time work counts.  SSR 96-8p.  The O*NET OnLine reports that 60% of counter and rental clerks work 40+ hours per week.  We now have the number of counter and rental clerks to less than 10,000.  That is less than a significant number of jobs in the Ninth Circuit, your circuit may vary.  

To address the 68,000 jobs reported by JBP, we used County Business Patterns and the Employment Projections nested inside of the Occupational Outlook Handbook.  CBP and OOH are listed sources as examples of administrative notice.  We answered the full-time question using the O*NET but could have used the answer provided by JBP.  

The answer of why JBP counts the subsector instead of the industry group or the specific industry rests on the structure used.  JBP relies on the OEWS to estimate the occupation-industry intersection.  Where the OEWS does not report 4 or 5-digit NAICS codes, JBP does not use them.  But our hands are not tied and we can "drill down."  We must "drill down."  Our client's future depends on it.  


Suggested Citation:

Lawrence Rohlfing, Storage-Facility Rental Clerk -- Using County Business Patterns to Crush the JBP Job Numbers Estimate, California Social Security Attorney (May 19, 2022) 

Wednesday, May 18, 2022

Job Browser Pro - Ver. 1.7.4 -- Example Small Products Assembler I

I have pointed out the double and triple counting of jobs in certain occupations in the DOT.  Last week, SkillTRAN released version 1.7.4.  Jeff Truthan gave me an advance copy at the NOSSCR conference and I took it for a spin.  Unfortunately, my laptop did not have the right video output to share on the Reverse Engineering session on Friday afternoon.  

In our unending quest for reliable job numbers, version 1.7.4 has eliminated the inconsistent use of three and four-digit NAICS codes in estimating job numbers.  It remains critical to ask the vocational witness, 
"What version of JBP are you using?"

The VW that subscribe to the online version always get the most recent version.  The VW that use the desktop version and use an outdated version that double counts, allows the insertion or deletion of industries, and other nefarious problems.  Ask, "What version of JBP are you using?"

As valuable as JBP is in identifying job numbers, the VW use of JBP does not end the inquiry, it is only the beginning.  JBP continues to use equal distribution of job numbers between DOT codes at the occupation-industry (SOC-NAICS) intersection.  With circa 13,000 DOT codes, estimating job numbers by the skill and exertion levels within the occupational group would constitute a Herculean undertaking.  

That leaves the cross-examination of the VW up to the intrepid representative.  Let's take a look at small products assembler I (DOT 706.684-022) as an example.  JBP continues to list industry groups within the food manufacturing subsector.  SkillTRAN does not explain its NAICS assignments.  The VW must explain why people that "mass produce small products" perform those function in animal food manufacturing.  Without a reasonable articulated basis, that industry assignment has an unresolved apparent conflict -- if and only if the JBP report is submitted into the record before the date of the ALJ decision.  

JBP ver. 1.7.4 estimates that small products assembler I represents 18,445 jobs.  Most of those jobs come in the temporary help services (NAICS 561320) industry.  The WT hyperlink lists five DOT codes, four light and unskilled, one heavy and semi-skilled.  But that is not the way that production workers pan out.  The ORS available on the CBP hyperlink and of course in tells us that production workers engage in medium work between 64% and 85% of the jobs.  None of the called out temp jobs require medium exertion.  Nor does the assignment of 80% of the jobs in the temporary help services industry to light unskilled work conform to the reality that most of the jobs in the occupation are semi-skilled.  A better set of assumptions are that most of the jobs in the occupation-industry intersection require medium exertion and at least half are skilled.  

JBP reports the 2018 and 2021 data sets for the ORS. gives the 2017, 2018, 2019, 2020, and the 2021 data sets.  The 2018 data set represents the final wave of the first set.  The 2020 data set is the first of the second wave.  The 2021 data set is the preliminary second wave data.  The 2018 data set reports that 53% of production workers are unskilled.  The 2020 data set describes production workers as engaged in unskilled work in 28.7% of jobs.  The 2021 data set does not set forth SVP data for production workers.  Whatever the percentages are, they are not 80% light and unskilled.  For small products assembler I, the estimate of 11,000 jobs in temporary help services is not reliable.  A substantial question also arises whether temporary help services constitutes sustained employment for purposes of substantial gainful activity under SSR 96-8p.  

Just because it is written down somewhere does not mean that our job is over.  Examine the data, the methodology, and ask questions like your client's life and quality of life depend on it.  It does.  


Suggested Citation:

Lawrence Rohlfing, Job Browser Pro - Ver. 1.7.4 -- Example Small Products Assembler I, California Social Security Attorney (May 18, 2022) 

Monday, May 16, 2022

Questions and Answers from Austin NOSSCR Conference 5/13/22

 Here are most of the unanswered questions from the conference:

1.  When VE's testify they use their own methodology that they will never proffer it because they claim it is work product and the judges never demand they proffer it.  How do you handle this?

Did I understand you correctly that the manner in which you calculate the number of jobs is unique to you and your organization?  

Has your methodology been subjected to peer review?

Does any other vocational witness use the same methodology?

Is your methodology a "well-accepted methodology" in the community of vocational witnesses?

If the vocational witness is not using a "well-accepted methodology" for extrapolating their local experience to the national economy, it is not substantial evidence.  Biestek v. Berryhill at 1155.  

2.  Does the addresser job actually exist?

Not in significant numbers.  See the OIDAP slide show at slide 7.  The OIDAP presentation on the SSA website lists addresser, tube operator, cutter-and-paster of press clippings, head host/hostess, and magnetic-tape winder as "doubtful that these jobs, as described in the DOT, currently exist in significant numbers in our economy."  

Look at the DOT description and make an apparent conflict argument supported by the OIDAP list.  

3. What the heck is non-confrontational supervision?  I get that in a lot of ALJ hypos.  

It is an opportunity to cross-examine.  The concept of non-confrontational supervision is an oxymoron.  A supervisor does not need to discuss something with a worker that is not confrontational -- something that is not obvious or apparent.  Non-confrontational is synonymous with superficial contact.  

What is it about accepting criticism from supervisors envisions the absence of confrontation?

4. Knocking the job numbers down until they are not significant. What is the lowest number that is still significant?

In the Ninth Circuit, the close call is 25,000 jobs.  Gutierrez v. Commissioner.  A number of unpublished cases dealing with SSR 00-4p conflict have addressed the residual number of jobs.  The ALJ must explain why the sub-25,000 number is still significant at least in the Ninth Circuit.  

5. In your experience are they any ALJs approving cases based on VE job numbers cross ex or is this purely a way to create an appealable issue for federal court?  

Some of the ALJs are listening and approving cases.  Many of the ALJs see this as an assault on the citadel.  Think Prosser and privity for torts.  Many cases will end up on court.  Whether we take it up personally or we hand it off to another attorney, it is the fiduciary duty of the attorney/representative in the hearing to create the record that the ALJ might accept or might need an appeal.  

6. Whenever I ask the VE their methodology, they say "I just told you."  How do you define methodology?  

Merriam Webster defines methodology:

Definition of methodology

1: a body of methods, rules, and postulates employed by a discipline : a particular procedure or set of procedures

demonstrating library research methodology

the issue is massive revision of teaching methodology

— Bob Samples

2: the analysis of the principles or procedures of inquiry in a particular field

My definition of a methodology is that it is understood and replicable.  If another trained person did exactly what you said, that person would get the same or similar result.  A methodology is not a bibliography and a black box.  

7.  Is "Document Preparer" actually a stand alone job? If it what universe are many of our clients, for non-medical "personal barriers to employment in a corporate setting" reasons actually eligible for these jobs?

Amy Shelton said at the conference that document preparer is now a scanner.  The OOH describes general office clerks, in part:

Some clerks file documents or answer phones; others enter data into computers or perform other tasks using software applications. They also frequently use photocopiers, scanners, fax machines, and other office equipment.

(Emphasis added).  Whether it is standalone, that is a challenge that needs to be raised.  My office is paper-less and I have one person that scan incoming mail and records as her sole duty.  The job requires use of the case management software and routing-tagging-tasking procedures.  It is standalone but might be semi-skilled.  

I am not clear on what personal barriers to work in a corporate setting means.  

8.  Can an usher do the job one handed? Can a school bus monitor do that job one handed?

Usher is probably part-time (about 90% of the jobs) and it requires standing/walking all day.  See the O*NET and ORS respectively.  The ORS says that a slim majority of the jobs permit one-handed gross and fine manipulation.  See the ORS.  Whether those factors overlap is a statistical/expert question.  

9. Some of the VE’s at our hearings will testify that in entry-level unskilled jobs an individual can be off-task up to and through 15% of the work-day. Others will say no more than 10%. What’s your opinion?

A good question for Amy Shelton.  

The reason that off-task behavior is tolerated in employment is simple - everyone is off-task some of the time.  What is not tolerated is additional off-task behavior.  The follow up question:  assume that the person is off-task as much as any other employee in the tolerable range and in addition to that amount of off-task behavior, the person is off-task an additional 5, 10, or 15% due to pain/mental illness/etc.  

People with impairments are not super employees aside from their impairments.  They are like everyone else.  

10.  Light unskilled with use of cane for ambulation.

A good question for Amy Shelton.  

Light work that does not require use of both hands for manipulation, it is conceivable that those jobs might be available.  If the person needs a cane or other assistive device to walk, the person probably has a limited duration of standing/walking during a workday.  Use the Occupational Requirements Survey to explore bilateral gross or fine manipulation and for the second issue of total time standing/walking during the workday.  

11. Light work, simple repetitive tasks with few, if any changes in work environment, occasional contact with supervisors, co-workers and public.

Few, if any, changes in a work environment requires that we look at the reasoning level of the occupations.  Reasoning level 1 requires "occasional or no variables in or from [standardized] situations encountered on the job."

Reasoning level 2 requires the ability to "deal with problems involving a few concrete variables in or from standardized situations."  

We start with the assumption the R2 requires more than R1 -- this is a hierarchy.  R2 requires more than occasional variables.  R1 can require up to occasional variables.  The full range of R1 does not fit the question.  The VW must identify R1 work and explain why that occupation does not have occasional variables.  

12. After submitting post hearing brief pointing out the VE's testimony being in conflict w the DOT, the ALJ schedules a supplemental hearing with another VE... how best to respond to that?

Wash, rinse, dry, and repeat the process.  Mark that down as a win for discrediting bad testimony and do it again.  

13.  If an individual has periods of severe open sores over a substantial part of their person, how would you phrase a vocational limitation that would include this issue?

I would first compare that person to the psoriasis or other dermatological listings.  

At steps 4 and 5:  Assume that the person cannot tolerate a work environment with any airborne pollutants, cannot tolerate an environment where the [body part] could be experience any physical pressure, and other limitations supported by the testimony or medical record.  

14.  Escort Driver - The V W. at a prior hearing described this job as sedentary. I argued it was light due to the use of foot pedals. Would the Ms. Shelton opine whether, as it is performed today, would it be classified as light (not sedentary).

Jeff Truthan, Amy Shelton, and I discussed escort vehicle driver before the session.  Jeff Truthan stated his opinion that the DOT misclassified escort vehicle driver because of the frequent or constant use of foot pedals.  Light work includes:
when it requires sitting most of the time but entails pushing and/or pulling of arm or leg controls
DICOT Appendix C.  

15.  Claimant limited to occasional interactions with supervisors and coworkers. ALJ tries to shut down questioning about claimant who may be able to interact that percentage of the time but not predictably, and not on the supervisor’s schedule. Similarly, issue that neither training nor corrections by supervisors constitute superficial interaction.

What is the data source for occasional interactions with supervisors and coworkers?  The O*NET is very helpful on this.  

A person that can interact occasionally cannot interact with others two-thirds of the day, can that person work?

SSA says that it is critical for unskilled employment to "work in coordination with or proximity to others without being (unduly) distracted by them."  Do you agree with that statement?  Why is this occupation different?

SSA says that it is critical for unskilled employment to "accept instructions and respond appropriately to criticism from supervisors."  Do you agree with that statement?  Why is this occupation different?

Use POMS DI 25020.010 Mental Limitations.  

16.  Did you ever ask the ve to repeat the judge hypothetical?

The witness cannot repeat what the witness did not say.  I ask the witness to tell me what that person heard.  

War story:  A judge asked a hypothetical question.  The VW responded.  I asked the ALJ to repeat the question to me thinking that I did not get it all.  The ALJ refused.  I asked the VW to tell me the hypo that the VW heard.  The ALJ allowed that question because it is proper cross-examination.  

17.  re parking lot attendant: can you explain how you got the occasional figures from ONET so quickly? What steps did you take to find that info for your questioning just now?

I use  I run an everything report for the DOT code cited.  I search in the report for "contact with others" or "group or team" and go to that data point.  I search for "40 hours" to look for full-time work statistics.  Those are in the O*NET.  I search for other job requirements in the same everything report.  

18.  How can someone be a nut sorter if they can only handle/feel/finger 2/3 of the workday? What do they do the other 1/3?

The DOT classifies nut sorter as frequent reaching, handling, and fingering - each.  It does not say that they handle and finger frequently in the aggregate.  

The ORS does not classify frequency of manipulation for inspectors, testers (SOC 51-9061).  The O*NET reports that inspectors, testers make repetitive motions less than continuously (< 67% of the day) in 83% of jobs.  

Finally, the DOT describes the occupation:
Removes defective nuts and foreign matter from bulk nut meats: Observes nut meats on conveyor belt, and picks out broken, shriveled, or wormy nuts and foreign matter, such as leaves and rocks. Places defective nuts and foreign matter into containers.
The occupation objective is to remove defective nuts and foreign matter.  The first listed task is observation.  Observation is more critical than places defective nuts and foreign matter into containers.  

19.  Does JBP provide numbers to a particular DOT code that are already reduced by the ORS data?

No.  That would represent a massive undertaking for 13,000 occupations and would create a superclass or subclass of occupations where some get treated differently than others.  That distinction would constitute a contradiction in process and provide us with a basis for jettisoning all the data.  


Suggested Citation:

Lawrence Rohlfing, Questions and Answers from Austin NOSSCR 5/13/22, California Social Security Attorney (May 16, 2022) 

Friday, May 13, 2022

Vocational Acronyms

Abbreviations, acronyms, and truncated expressions are the shorthand for communication.  Here is a list of some of acronyms used during examination of vocational witnesses.  

DOT - DICOT - Dictionary of Occupational Titles, 4th ed. rev. (Dept. of Labor 1991)..  

SCO - Selected Characteristics of Occupations Defined in the Revised Dictionary of Occupational Titles (Dept. of Labor 1993).

RHAJ - Revised Handbook for Analyzing Jobs (Dept. of Labor 1991).

CBP - County Business Patterns (Census 2020).

OOH - Occupational Outlook Handbook (Dept. of Labor 2021).

EP – Employment Projections (Dept. of Labor 2021).

OES - Occupational Employment Statistics (Dept. of Labor 2021) (obsolete)

OEWS - Occupational Employment Wage Statistics (Dept. of Labor 2022)

NAICS – (pronounced NAKES) - North American Industry Classification System (Census 2022). 

O*NET - Occupational Information Network (Dept. of Labor 2022)

ORS - Occupational Requirements Survey (Dept. of Labor 2021)


Suggested Citation:

Lawrence Rohlfing, Vocational Acronyms, California Social Security Attorney (May 13, 2022)