Transitions for Planning Departments
Paul C. Zucker
Copyright 1997 Zucker
Planning departments throughout the country are being reinvented and going through major transitions. One of the techniques being used to aid this transition is organizational, management or performance audits. This can be frightening or stimulating - depending on your attitude and the status of your organization. This paper is designed to help planners prepare for and cope with an audit.
WHO ASKED FOR THIS THING ANYWAY?
Audits generally are initiated by one of three sources.
1. Routine
Some organizations are required to have a management audit every 4 or 5 years. These are normally state or regional agencies or local agencies that were established through special state legislation. For example, we recently completed an audit for a Florida Commission that was established by State legislation and is required to have a performance audit every four years.
A number of communities are routinely conducting performance audits on all of their departments. We've seen or participated in recent audits of this type in Kentucky, Arizona and Utah.
2. Director Initiated
One of the best kinds of performance audits is that initiated by the department director. The attitude is, "We're good but can be better." This corresponds to the current management idea of continuous improvement. Often these audits are more limited. They may look at one division or one function such as development review or building permits. We performed this type of work for Santa Cruz County, California; and Columbus, Indiana.
3. Troubled Organizations
The most common types of audits we see are for "troubled organizations." Generally, these organizations have fallen out of favor with the city manager, mayor, or elected officials. Often the director of the department is in trouble. In some cases the director recently left and the audit is used to set a new direction or provide advice for a new director or restructuring. In some cases, the problems have been going on for as long as ten years, and finally, someone decides they can't continue any longer.
If yours is a troubled organization, your chances of success and stabilization of a bad situation may be better if you pursue the director initiated audit rather than have the troubled organization audit placed on your desk.
WHAT SHOULD YOU DO?
Don't panic, irrespective of who asked for the audit, we suggest the following:
1. Be candid and open
In the final analysis, the goal is to improve your organization. The auditor can be most helpful when you are candid and open in your conversation. A good auditor will quickly determine if you are not being straight forward and this may reflect on you in the audit.
2. Don't try to control the auditor
We've had directors call ahead to people we were about to interview to tell them what to say or not to say. We've had directors try to manipulate who would or would not be interviewed. Once the auditor realizes that this is taking place, he is likely to dig even deeper to find the people or situations you are trying to hide.
3. Don't try to control staff
Staff needs to be very candid in these conversations with the auditor and not be afraid to share opinions on a confidential basis. The auditor will quickly sense that you are trying to control or inhibit staff and it will not work to your favor.
4. The organization should be itself
Don't try to put on an unreal or good front just for the auditor. If you truly want to improve, the auditor needs to see you -- warts and all. We've had organizations sponsor clean up days before our arrival, write operations manuals that are never executed, etc. The auditor will hear about all of this during the audit.
5. Use the audit to forward your agenda
The auditor will be looking for good ideas to add to their audit so share yours with the auditor. An audit is a good way to accomplish things that you have been unable to accomplish. We always tell our clients that we're willing to be used that way. Obviously, we need to agree with your idea using our independent analysis. However, without your help or suggestion, the idea may not have surfaced.
6. See criticism as gifts
Studies show that on the average only 4% of people who have problems with an organization complain to the organization, the other 96% bad mouth the organization to 8 or 12 other people. In spite of most peoples' assumptions to the contrary, it is very difficult to obtain criticism that can be used for improvement ideas. Therefore, every criticism in an audit should be viewed as a precious gift to the organization.
WHO DOES AUDITS - HOW ARE AUDITORS SELECTED?
Most auditors are selected through an RFP bid process and these auditors are private consultants. Some cities and counties have their own independent audit departments. For specialized audits, these departments at times also bring in sub-consultants to assist. These consultants are often selected sole source but may also be selected by RFP. For example, we performed this role in an audit in Nevada for a development review department.
Some states also have special audit departments. For example, we recently bid an audit for the Tahoe Regional Planning Agency, that was eventually given to the California Bureau of State Audits, a state agency.
There are a variety of firms conducting audits. We tend to group them in basic groups as follows:
- The big six accounting firms.
- Mid to large size national firms that specialize in governmental contracts.
- Small firms, often localized and sometimes specializing in specific topics such as fire, water or planning.
- Firms that are primarily technical, i.e., engineering or planning, that occasionally bid on audits.
- Other CPA firms.
We've had the opportunity to work with all five types of firms, either as a private consultant or as a sub-consultant. The success of all the firm is highly dependent on the specific staff they assign to the audit. The big six accounting firms are often selected for large audits or for audits requiring a political public image. We've found their work to be spotty. The senior level people are often only peripherally involved and the work is turned over to new MBAs. Billing rates are generally higher than other firms so the amount of detail may be less than in other audits.
The mid to large size national firms can produce excellent work, but to some extent, it also depends on the person they assign to the audit. In our experience, the people they assign are generally more senior and experienced than the big six accounting firms. They also often use specialized sub-consultants and seem to have a large resource of specialized people to select from.
The small more localized firms can be very good and should be easy to reference check.
Firms that are primarily technical in a field, often do not know how to do an audit. However, they can be excellent as sub-consultants to a trained auditor. For example, when we conducted an audit for a California grading department, we used as a sub-consultant a small San Diego County engineering firm whose principal had also been the director of several governmental grading programs.
Other CPA firms are difficult to assess. Most are good at doing financial audits, but seem to have limited experience in organizational and management audits. The use of CPA firms can be particularly tricky due to the standards set forth in the United States General Accounting Office guideline for audits, generally referred to as "The Yellow Book," due to the color of its cover.
The Yellow Book sets forth standards for both the audit and the auditors. The standards for the audits are followed by most firms that conduct audits. However, the standards for auditors are another matter. The Yellow Book establishes specific continuing education requirements as well as external quality control review requirements. Most non-CPA firms would not meet these requirements. While to date this has not been much of a national problem, we have run into the issue in Florida. If carried to an extreme, this would set up an interesting paradox. The firms producing the best audits would not be authorized to do so. The firms producing the poorer audits would be fully certified.
WHAT SIZE OF ORGANIZATION DO AUDITS?
We've conducted audits for all sizes of cities and counties with a population of 20,000 being the smallest and 3.4 million the largest. In theory, any size organization could benefit from an audit.
HOW LONG DO AUDITS TAKE?
Typically audits take from 3 to 6 months to complete, although we've done a few in one or two months. Seldom do audits exceed 6 months. For most operations, a 4 or 5 month timeline should be sufficient.
WHAT DO THEY COST?
The costs of an audit can very substantial depending upon the RFP and the organization. Factors that have an impact on the cost include:
- Will specialized sub-consultants be required, for example for automation?
- How detailed an analysis of staffing needs is desired?
- How much customer input is needed?
We've conducted audits for as little as $5,500 and as high as $220,000. For small to mid-sized organizations (10-25 employees) a budget of $30-$35,000 could be reasonable. Mid to large sized organizations (25-50 employees) a budget of $50-$60,000 could be expected. Larger or more complex organizations would require larger budgets.
HOW MANY RECOMMENDATIONS TO EXPECT?
The number of recommendations per audit will vary widely by the nature of the organization and the style of the audit. We tend to provide more recommendations than many auditors and our audits have averaged 105 recommendations per audit with a range of 20 to 267.
METHODOLOGY/TECHNIQUES
The methodology and techniques used in the audit may vary depending upon the approach of the auditors and the needs of the organization being audited. However, most audits will include all or some of the following tasks.
1. Interviews
Most audits will include numerous interviews including:
- Policy makers: Policy makers (i.e., elected officials, planning commission) are often interviewed one-on-one in a confidential setting. We find that they will often tell us things in this setting that they wouldn't say in a group meeting. Some auditors interview the entire group but we have not found this particularly useful.
- Customers: Selected one-on-one confidential interviews with customers are often used. We find this less effective than customer focus groups because they often deal only with a specific incident or perspective and lack the dynamics and synergism that seems to take place in focus groups. Often the policy makers suggest names for one-on-one interviews and in these instances we generally interview.
- External Managers: Interview with managers and staff of other departments or organizations that impact planning are generally conducted. For example these would include building, engineering, data processing, personnel, etc.
- Internal Managers: All the managers of the department being audited are normally interviewed as well as mid-level managers and supervisors. Many of these might be interviewed more than once.
- Staff: For small organizations, each staff member is interviewed. For large organizations we interview only selected staff, however, a few auditors we know want to interview all staff.
2. Employee Questionnaires
Most audits use one or more survey instruments. These can be particularly valuable because they can provide objective data to support other audit findings. Typical surveys include:
- Anonymous Employee: Most auditors use some form of anonymous employee surveys that rate the employee's satisfaction on a variety of factors. We generally use a relatively short, 39 question, survey where questions are rated on a scale of 1-7 from Totally Disagree to Totally Agree. The questionnaire can be completed in 10 minutes. A few sample questions are:
- When problems are identified, we move quickly to solve them.
- We have a strong emphasis on training.
- Management discuss objectives, programs and results with employees regularly.
Some auditors use a more extensive questionnaire. We recently worked with an auditor who uses a 78 question instrument that not only measures satisfaction but also urgency of the item and probability for improvement. A sample question is:
The amount of recognition you receive for your work/accomplishments:
(a) How satisfied are you with this?
0 1 2 3 4 5 6 7 8 9 10
(b) How urgent is an improvement in this?
0 1 2 3 4 5 6 7 8 9 10
(c) Probability that this will be improved?
0 1 2 3 4 5 6 7 8 9 10
For larger organizations, all of these questions are normally sorted by Departmental Division or type of employee (i.e., clerical, planner, etc.)
There is a real advantage if the auditors use the same questions repeatedly in numerous audits. Used only once, it may be difficult to determine the meaning of a specific answer, but used repeatedly, scores from one organization can be compared to another that can help the interpretation of the response.
- Confidential But Not Anonymous Employee Questionnaires: We often use a relatively long, open ended employee questionnaire of 40-50 questions. These might take an hour to complete. Their disadvantage is that they can be very costly to analyze. However, they often produce topics for analysis that are otherwise not uncovered. We keep these confidential, but not anonymous, so that we can follow up comments with the specific employee as appropriate.
Two typical questions would be:
- What important services or programs have been cut from your Department or division or are not being pursued due to budget or other considerations that you feel should be added?
- Do you feel any of the ordinances, policies or plans in relation to the planning or development function should be changed? If so, list them and explain why.
3. Customer Surveys
Surveys are often mailed to customers of the organization. For example, a survey might be sent to the last 100 people who applied for a certain type of permit. As with other questionnaires, using a similar survey from community to community can help in the analysis of the response. We typically use a 25 question instrument asking a response ranging from Strongly Agree to Strongly Disagree. The questionnaire can be completed in 10 minutes and we typically receive a 30-40% response. Typical questions include:
- In general, Planning Commission staff have dealt with me in a positive "Here's what needs to be done to get the application approved" approach rather than a negative "You can't do that" approach.
- I found the handouts supplied by the Planning Commission to be useful and informative in explaining the requirements I had to meet.
At the end of the questionnaire we ask for comments or suggestions. Often the questionnaires provide good suggestions for areas needing further research.
4. Focus Groups
A good way to obtain customer input is by professionally run focus groups. Where budgets allow, we use from 3 to 10 focus groups. Each group is segmented by specific topics or participants, i.e., we might have engineers and architects in one group, developers in one, and citizens in another. Each group of 8-10 people meets for 2 or 3 hours.
5. Peer Review
If budgets are sufficient, we like to include peer review in our audits. Many auditors do not use peer panels. Under peer review 3 to 6 managers from other organizations spend 2 or 3 days examining the organization being audited. Peer members must be carefully selected to meet the audit needs. Topics for peer panel consideration need to be narrowed, given the short amount of time. We've found peer panels can reinforce other aspects of the audit and moreover raise issues that the auditor may have missed.
6. Benchmarking
Some, but not all, audits require benchmarking or comparison with another community. While useful for political reasons, there can be numerous problems with community comparisons. Comparisons often unintentionally compare apples to oranges or the circumstance in one community is simply not comparable to another. When we do community comparison, we try to limit the comparisons to very specific items such as the amount of time requested to approve a sub-division, etc.
7. Documentation and Analysis
Most of the studies described so far are used as background for more detailed analysis. The work of auditors in the documentation and analysis phase seems to vary substantially. We use an interactive technique.
A variety of external inputs are used as previously described. The operational analysis compares records, interviews and observations. Studies show that any one of these taken along can be as much as 50% inaccurate. Therefore, we use records to check on interviews and observations. Interviews are used to check records and observations. Observations are used to check records and interviews.
Our analysis is often grouped by topics as follows:
- Mission and Policy: The audit must be placed in the context of the organization's mission and legal standing. In a planning agency this normally requires a review of state legislation, local charters and by-laws, mission statements, operation manuals and policy documents. The policy documents include the comprehensive plan and ordinances. Although audits are normally not designed to evaluate these documents in a detailed way, we find it necessary to at least examine the extent that they may be creating customer's service problems.
- Process: The process analysis looks at the various processes in the organization. Normally there is not enough budget to flow chart or measure all the processes so a few key ones are selected.
- Organizational Issues: A variety of organizational issues are reviewed. The list of topics and depth of review is often dependent on findings from the interview and questionnaires. Normally, budgets will not allow a full review of each. These topics include:
- Organizational pattern: Layers in the organization, spans of control.
- Personnel Issues: Work loads, management communication, training. Some limited aspects of pay and classification may be examined, but audits seldom include what are known as pay and classification studies.
- Records: Records retention policy, records systems, records problems.
- Automation: These days virtually all audits include some review of the automation systems. Generally, it is known in advance or as part of the RFP if a detailed review is requested for various systems such as GIS or permitting systems.
- Office Space and Equipment: This topic varies substantially by audit. Some, for example, may specify a review of a one-stop-shop system or the telephone system.
FINDINGS
Findings, of course, vary substantially from audit to audit. Typical problems we have uncovered in past audits include:
Lack of a clear mission
The mission statements that exist are typical government bureaucratize are not understood by staff, and don't represent the mission of the organization as practiced.
Confusion as to who the customer is and no customer service strategy.
Generally there is a customer comment card but this tends to be the extent of customer feedback. There is no well thought out customer program.
Poor Systems
The organization has not arranged its work so the routine follows a systematic flow.
Few measurement or monitoring systems
The organization has not set targets for activities and therefore cannot measure or monitor them. In some cases targets have been set but they are not measured.
Managers performing operational tasks while management tasks go unattended.
Managers often think they are helping when doing operational tasks. This is true if their management tasks have been completed; if not, the organization suffers due to lack of management.
Lack of training budgets or training program.
Generally the budget for training is totally inadequate. In a few cases we've seen a good training budget but no training program to go with it.
Poor coordination with other departments.
Slow permitting is often caused by other departments. However the planners have not taken aggressive action to solve this problem.
Poor phone system or inadequate training in how to use the system.
Increasingly we find contemporary phone equipment but it has not been programmed to meet the department's needs. Training on features of the system is often incomplete. Finally the improper use of voice mail or unreturned phone calls creates customer service problems.
Outdated computers or programs and lack of integrated systems.
Some departments have not yet installed integrated permit issuance systems. Additionally, it is not uncommon to find manual logs where a simple spreadsheet or data base program would be useful.
Top heavy in organization and too many layers.
As departments have been cut back, the cutbacks have left the organization top heavy. Also the new themes of removing layers, and increasing span of control have not been implemented.
Poor filing systems.
We often see multiple uncoordinated filing systems, poor use of out cards, slowness in microfilming or no microfilming programs, and poor rules as to what gets filed, plus no retention policy.
Poor management
We've seen few organizations with employee problems, but many organizations with management problems.
Comprehensive plans and ordinances.
Plans and ordinances are often out of date, are too voluminous, lack a clear format, have amendments that are not integrated, do not include an update subscription service, and have no responsibility for up-dating.
Slow timelines and confusion on permit issuance
There is a lack of pre-set timelines for various activities and no measurement system to monitor timelines.
Directions
We have never been asked to comment on specific managers nor would we accept audits designed to fire a manager. However, it is not unusual that organizations have management problems. I believe in a few cases our audits have helped salvage a director who was in trouble. In a few cases directors have left right before or during the audit. In two cases directors were terminated a year after our audit was completed.
Effectiveness vs. Efficiency
Although many RFPs talk about both effectiveness and efficiency, in reality most audits primarily deal with efficiency. The budgets are rarely adequate to do a true effectiveness audit.
Boards and Commissions
Problems include getting items on agendas, tendency to continue items from meeting to meeting, lack of findings, minutes not up-to-date, no consent calendar, little or no training.
Handouts
Poor or non-existent or out of date.
Facilities
Lack of parking, poor or misleading signs, no co-location of permit functions.
REAL LIVE HORROR STORIES FROM OUR AUDITS
To end on a light note, the following stories are taken from some of our audits.
Plan Jump (New dance craze)
In one building department we saw secretaries jumping up and down to fold thick sets of building plans. When asked what they were doing, we were told they were folding them to fit into file drawers since the cabinet for rolled plans was full. Not only was this time consuming but the plan checkers didn't like all the creased plans when they were doing their review. Our recommendation -- buy another plan rack for rolled plans.
The Infamous Leroy Machine
We've observed this in two audits. Draftsmen were using Leroy machines for applications that would be completed better and faster using the computer and "sticky back." In both cases Macintosh Computers were sitting idle.
Money Money Money
Fifteen million dollars in housing re-hab funds was about to be lost because purchasing wouldn't process the paper work on time.
Lend Me Your White Out
Out of date forms and handouts were still be distributed.
Blank Forms.
We found a blank 3 page form stapled to every permit in the file. Staff told us the form was useless but management insisted it be used.
Equipment Goofs
On numerous occasions we have found recent acquisition of telephone equipment, office copiers or computers that were the wrong product for the organizations needs.
Data/Data/Data
We found three inches of computer printouts of management data, that was being produced on a monthly basis, and managers never looked at the data.
Implementation Follies
Plan after plan is adopted but never implemented, or if implemented, is never tested to see if the expected results were achieved or if anyone cared.
Paul C. Zucker is President of Zucker Systems, a planning and management consulting firm located in San Diego, California. The firm specializes in advising planning and community development departments. Mr. Zucker is the author of a major new book on managing planning departments, The ABZs of Planning Management and publishes a national subscription service newsletter for planning departments, Z Management Ideas. He can be reached at 1-800-870-6306.