General Expectations

Projections of losses, revenues, and expenses under hypothetical stressed conditions serve as the fundamental building blocks of the pro forma financial analysis supporting enterprise-wide scenario analysis. BHCs should have stress testing methodologies that generate credible estimates that are consistent with assumed scenario conditions. It is important for BHCs to understand the uncertainties around their estimates, including the sensitivity of the estimates to changes in inputs and key assumptions. Overall, BHCs’ estimates of losses, revenues, and expenses under each of the scenarios should be supported by empirical evidence, and the entire estimation process should be transparent and repeatable. The Federal Reserve generally expects BHCs to use models or other quantitative methods as the basis for their estimates; however, there may be instances where a management overlay or other qualitative approaches may be appropriate due to data limitations, new products or businesses, or other factors. In such instances, BHCs should ensure that such processes are well supported, transparent, and repeatable over time.

Establishing a Quantitative Basis for Enterprise-Wide Scenario Analysis

Generally, BHCs should develop and use internal data to estimate losses, revenues, and expenses as part of enterprise-wide scenario analysis. 29 However, in certain instances, it may be more appropriate for BHCs to use external data to make their models more robust. For example, BHCs may lack sufficient, relevant historical data due to factors such as systems limitations, acquisitions, or new products. When using external data, BHCs should take care to ensure that the external data reasonably approximate underlying risk characteristics of their portfolios, and make adjustments to modeled outputs to account for identified differences in risk characteristics and performance reflected in internal and external data.

BHCs can use a range of quantitative approaches to estimate losses, revenues, and expenses, depending on the type of portfolio or activity for which the approach is used, the granularity and length of available time series of data, and the materiality of a given portfolio or activity. While the Federal Reserve does not require BHCs to use a specific estimation method, each BHC should estimate its losses, revenues, and expenses at sufficient granularity so that it can identify common, key risk drivers and capture the effect of changing conditions and environments. For example, loss models should be estimated at a sufficiently granular subportfolio or segment level so that they can capture observed variations in risk characteristics and performance across the subportfolios or segments and across time, and account for changing exposure or portfolio characteristics over the planning horizon.

While BHCs often segment their portfolios and activities along functional areas, such as by line of business or product type, the leading practice is to determine segments based on common risk characteristics (e.g. credit score ranges or loan-to-value ratio ranges) that exhibit meaningful differences in historical performance. The granularity of segments typically depends on the type, size, and composition of the BHC’s portfolio. For example, a more diverse portfolio—both in terms of borrower risk characteristics and performance—would generally require a greater number of segments to account for the heterogeneity of the portfolio. However, when segmenting portfolios, it is important to ensure that each risk segment has sufficient data observations to produce reliable model estimates.

As a general practice, BHCs should separately estimate losses, revenues, or expenses for portfolios or business lines that are sensitive to different risk drivers or sensitive to risk drivers in a markedly different way. For instance, losses on commercial and industrial loans and commercial real estate (CRE) loans are, in part, driven by different factors, with the path of property values having a more pronounced effect on CRE loan losses. Similarly, although falling property value affects both income-producing CRE loans and construction loans, the effect often differs materially due to structural differences between the two portfolios. Such differences can become more pronounced during periods of stress. BHCs with leading practices have demonstrated clearly the rationale for selecting certain risk drivers over others. BHCs with lagging practices used risk drivers that did not have a clear link to results, either statistically or conceptually.

Many models used for stress testing require a significant number of assumptions to implement. Further, the relationship between macroeconomic variables and losses, revenues, or expenses could differ considerably in the hypothetical stress scenario from what is observed historically. As a result, while traditional tools for evaluating model performance (such as comparing projections to historical out-of-sample outcomes) are still useful, the Federal Reserve expects BHCs to supplement them with other types of analysis. Sensitivity analysis is one tool that some BHCs have used to test the robustness of models and to help model developers, BHC management, the board of directors, and supervisors identify the assumptions and parameters that materially affect outcomes. Sensitivity analysis can also help ensure that core assumptions are clearly linked to outcomes. Using results from different estimation approaches (challenger models) as a benchmark is another way BHCs can gain greater comfort around their primary model estimates, as the strengths of one approach could potentially compensate for the weaknesses of another. When using multiple approaches, however, it is important that BHCs have a consistent framework for evaluating the results of different approaches and supporting rationale for why they chose the methods and estimates they ultimately used.

In certain instances, BHCs may need to rely on third-party models—for example, due to limitations in internal modeling capacity. In using these third-party models (vendor models or consultant-developed models), BHCs should ensure that their internal staff have working knowledge and a good conceptual understanding of the design and functioning of the models and potential model limitations so that management can clearly communicate them to those governing the process. An off-the-shelf vendor model often requires some level of firm-specific analysis and customization to demonstrate that it produces estimates appropriate for the BHC and consistent with scenario conditions. Sensitivity analysis can be particularly helpful in understanding the range of possible results of vendor models with less transparent or proprietary elements. Importantly, all vendor and consultant-developed models should be validated in accordance with SR 11-7 guidelines. 30

Some BHCs generated annual projections for certain loss, revenue, or expense items and then evenly distributed them over the four quarters of each year. This practice does not reflect a careful estimate of the expected quarterly path of losses, net revenue, and capital, and thus is only acceptable when a BHC can clearly demonstrate that the projected item is highly uncertain and the practice likely results in a conservative estimate.

Qualitative Projections, Expert Judgment, and Adjustments

While quantitative approaches are important elements of enterprise-wide scenario analysis, BHCs should not rely on weak or poorly specified models simply to have a modeled approach. In fact, most BHCs use some forms of expert judgment for some purposes—generally as a management adjustment overlay to modeled outputs. And BHCs can, in limited cases, use expert judgment as the primary method to produce an estimate of losses, revenue, or expenses. BHCs may use a management overlay to account for the unique risks of certain portfolios that are not well captured in their models, or otherwise to compensate for specific model and data limitations. Material changes in BHCs’ businesses or limitations in relevant data may lead some BHCs to rely wholly on expert judgment for certain loss, revenue, or expense projections. In using expert judgment, BHCs should ensure that they have a transparent and repeatable process, that management judgments are well supported, and that key assumptions are consistent with assumed scenario conditions.

As with quantitative methods, the assumptions and processes that support qualitative approaches should be clearly documented so that an external reviewer can follow the logic and evaluate the reasonableness of the outcomes. 31 Any potential shortcomings should be investigated and communicated to decisionmakers. In addition, any management overlay or qualitatively derived projections should be subject to effective review and challenge. BHCs should evaluate a range of potential estimates and conduct sensitivity analysis for key assumptions used in the estimation process. For example, if a BHC makes extensive adjustments to its modeled estimates of losses, revenue, and expenses, the impact of such adjustments should be quantified relative to unadjusted estimates, and these results should be documented and made available to BHC management and the board of directors. Finally, extensive use of management judgment to adjust modeled estimates should trigger review and discussion as to whether new or improved modeling approaches are needed. In reporting to the board of directors, management should always provide both the initial results and the results after any judgmental adjustments.

Conservatism and Credibility

Given the uncertainty inherent in a forward-looking capital planning exercise, the Federal Reserve expects BHCs to apply generally conservative assumptions throughout the stress testing process to ensure appropriate tests of the BHCs’ resilience to stressful conditions. In particular, BHCs should ensure that models are developed using data that contain sufficiently adverse outcomes. If a BHC experienced better-than-average performance during previous periods of stress, it should not assume that those prior patterns will remain unchanged in the stress scenario. BHCs should carefully review the applicability of key assumptions and critically assess how historically observed patterns may change in unfavorable ways during a period of severe stress for the economy, the financial markets, and the BHC.

In the context of CCAR loss and revenue estimates, BHCs should generally include all applicable loss events in their analysis, unless a BHC no longer engages in a line of business or its activities have changed such that the BHC is no longer exposed to a particular risk. BHCs should not selectively exclude losses based on arguments that the nature of the ongoing business or activity has changed—for example, because certain loans were underwritten to standards that no longer apply or were acquired and, therefore, differ from those that would have been originated by the acquiring institution.

Similarly, BHCs should not rely on favorable assumptions that cannot be reasonably assured to occur in stressed environments given the high level of uncertainty around market conditions. BHCs should also not assume any foresight of scenario conditions over the projection horizon beyond what would reasonably be knowable in real-life situations. For example, some BHCs have used the path of stress scenario variables to make optimistic assumptions about possible management actions ex ante in anticipation of stressful conditions, such as preemptively rebalancing their portfolios or otherwise adjusting their risk profiles to mitigate the expected impact. In the event of a downturn, the future path or progression of economic and market conditions would not be clearly known, and this uncertainty should be reflected in the capital plans.

Documentation of Estimation Practices

The Federal Reserve expects BHCs to clearly document their key methodologies and assumptions used to estimate losses, revenues, and expenses. 32 BHCs with stronger practices provided documentation that concisely explained methodologies, with relevant macroeconomic or other risk drivers, and demonstrated relationships between these drivers and estimates. Documentation should clearly delineate among model outputs, qualitative overlays to model outputs, and purely qualitative estimates. 33 BHCs with weaker practices often had limited documentation that was poorly organized and that relied heavily on subjective management judgment for key model inputs with limited empirical support for and documentation of these adjustments.

Loss-Estimation Methodologies

In this Section:

As noted earlier, a BHC’s internal stress testing processes should be designed to capture risks inherent in its own exposures and business activities. Consistent with any good modeling practices, when developing loss-estimation methodologies, BHCs should first determine whether there is a sound theoretical basis for macroeconomic and other explanatory variables (risk drivers) used to estimate losses, and then empirically demonstrate that a strong relationship exists between those variables and losses. For example, most BHCs’ residential-mortgage loss models used some measure of unemployment and a house price index as explanatory variables, which affect a borrower’s ability and incentive to repay.

Beyond the core set of macroeconomic variables that typically represents a given scenario, such as gross domestic products (GDP), unemployment rate, Treasury yields, credit spreads, and various price indices, BHCs often project additional variables that have a more direct link to particular portfolios or exposures. Some examples of these variables include regional macroeconomic variables that better capture the BHC’s geographic exposures and sector-specific variables, such as office vacancy rates and corporate profits. Using these additional variables to estimate the model can enhance the sensitivity of loss estimates to a given scenario and also improve the overall fit of the model. Any models used to produce additional risk drivers are key components of the loss-estimation process and, therefore, should be included in BHCs’ model inventories and receive the same model risk-management treatment as core loss-estimation models.

Generally, BHCs sum up losses from various portfolios and activities to produce aggregate losses for the enterprise-wide scenario analysis. BHCs should have a repeatable process to aggregate losses, particularly when they transform model estimates to combine disparate risk measures (such as accounting-based and economic loss concepts), different measurement horizons, or otherwise dissimilar loss estimates.

BHCs with leading practices used automated processes that showed a clear audit trail from source data to loss estimation and aggregation, with full reconcilement to source systems and regulatory reports and mechanisms requiring approval and logging of judgmental adjustments and overrides. These systems often leveraged existing enterprise-wide financial and regulatory consolidation processes.

BHCs with lagging practices exhibited a high degree of manual intervention in the aggregation process, and applied aggregate-level management adjustments that were not transparent or well supported.

Retail and Wholesale Credit Risk

BHCs used a range of approaches to produce loss estimates on loans to retail and corporate customers, often using different estimation methods for different portfolios. This section describes the observed range of practice for the methods used to project losses on retail and wholesale loan portfolios.

Data and Segmentation

Sources of data used for loss estimation have often differed between retail and wholesale portfolios. Due to availability of a richer set of retail loss data, particularly from the most recent downturn, BHCs generally used internal data to estimate defaults or losses on retail portfolios and only infrequently used external data with longer history to benchmark estimated losses on portfolios that had more limited loss experience in the recent downturn. For wholesale portfolios, some BHCs supplemented internal data with external data or used external data to calibrate their models due to a short time series (5-10 years) that included only a single downturn cycle.

BHCs with stronger practices accounted for dynamic changes in their portfolios, such as loan modifications or changes in portfolio risk characteristics, and made appropriate adjustments to data or estimates to compensate for known data limitations (including lack of historical periods of stress).

BHCs with weaker practices failed to compensate for data limitations or adequately demonstrate that external data reasonably reflect the BHC’s actual exposures, often failing to capture geographic, industry, or lending-type concentrations.

The level of segmentation used for modeling varied depending on the type and size of portfolio and estimation methods used. For example, BHCs often segmented the retail portfolio based on some combinations of product; lien position; risk characteristics such as credit score, loan-to-value ratio, and collateral; and underlying collateral information (e.g. single-family home versus condominium), though some models were estimated at the loan-level and others at the portfolio level.

BHCs with stronger practices had segmentation schemes that were well supported by the BHC’s data and analysis, with sufficient granularity to capture exposures that react differently to risk drivers under stressed conditions.

BHCs with weaker practices used a single model for multiple portfolios, without sufficiently adjusting modeling assumptions to capture the unique risk drivers of each portfolio. For example, in estimating losses on wholesale portfolios, these BHCs did not adequately allow for variation in loss rates commonly attributed to industry, obligor type, collateral, lien position, or other relevant information.

Common Credit Loan Loss-Estimation Approaches

BHCs have used a wide range of methods to estimate credit losses, depending on the type and size of portfolios and data availability. These methods can be based on either an accounting-based loss approach (that is, charge-off and recovery) or an economic loss approach (that is, expected losses). BHCs have flexibility in selecting a specific loss or estimation approach; however, it is important for BHCs to understand differences between the two loss approaches, particularly in terms of the timing of loss recognition, and to account for the differences in setting the appropriate level of reserves at the end of each quarter.

Expected Loss Approaches

Under the expected loss approach, losses are estimated as a function of three components—probability of default (PD), loss given default (LGD), and exposure at default (EAD). PD, LGD, and EAD can be estimated at a segment level or at an individual loan level, and using different models or assumptions. In general, BHCs used econometric models to estimate losses under a given scenario, where the estimated PDs were conditioned on the macroeconomic environment and portfolio or loan characteristics. Some BHCs used other approaches, such as rating transition models, to estimate stressed default rates as part of an expected loss framework.

BHCs with leading practices were able to break down losses into PD, LGD, and EAD components, separately identifying key risk drivers for each of those components, though they typically did not demonstrate this level of granularity consistently across all portfolios. For certain wholesale portfolios, some BHCs used long-run average PD, LGD, and EAD for a particular segment, such as a rating grade, to estimate losses. By design, estimates based on long-run average behavior over a mix of conditions, including periods of economic expansion and downturn, are not appropriate for projecting losses under stress and should not be used for these purposes.

BHCs with leading practices clearly tied LGD to underlying risk drivers, accounted for collateral and guarantees, and also incorporated the likelihood of a decline in collateral values under stress. However, most BHCs have more limited data on LGD and, as a result, BHCs often applied a simple, conservative assumption (e.g. 100 percent LGD for credit cards), based stressed LGD on their experience during the crisis, or scaled up the historical average LGD using expert judgment. In using such methods, it is important for BHCs to ensure that the process is well supported and transparent in line with the Federal Reserve’s general expectation for expert judgment-based estimates. Wherever possible, BHCs should benchmark their estimates with external data or research and analysis.

BHCs with lagging practices modeled LGD using a weighted-average approach at an aggregate portfolio level, without some level of segmentation (e.g. by lending product, priority of claim, collateral type, geography, vintage, or LTV). Or, they failed to demonstrate that LGD estimates were consistent with the severity of the scenario.

Although some BHCs found a relationship between EAD and credit quality, most BHCs did not model EADs to vary according to the macroeconomic environment, in large part due to data limitations. Rather, many BHCs applied a static assumption to estimate stressed EAD.

BHCs with stronger practices included the use of loan equivalent calculations (i.e. estimated additional drawdowns as a percentage of unused commitments, which are added to the outstanding or drawn balance) and credit-conversion factors (i.e. additional drawdowns during the period leading up to default—usually one year prior—as a percentage of both drawn and undrawn commitments) to capture losses associated with undrawn commitments.

BHCs with weaker practices did not project stressed exposures associated with undrawn commitments and/or relied on the assumption that they can actively manage down committed lines during stress scenarios.

Rating Transition Models

Many BHCs have used a rating transition-based approach to produce a stressed rating transition matrix for each quarter, which is then used to estimate losses for their wholesale portfolios under stress. These approaches used credit ratings applied to individual loans by the BHC and projected how these ratings would change over time given the macroeconomic scenario. Although the details of techniques used to link rating transitions to scenario conditions varied across firms, the process usually involved the following steps: (1) converting the rating transition matrix into a single summary measure; (2) estimating a time-series model linking the summary measure to scenario variables; (3) projecting the summary measure over the nine-quarter planning horizon, using the parameter estimates from the time-series model; and (4) converting the projected summary measure into a full set of quarterly transition matrices. BHCs using such an approach should be able to demonstrate that the summary measure responds to changes in economic conditions as expected (that is, worsens as the economic condition deteriorates) and results in projected rating transition matrices that are consistent with the severity of scenario. Judgmentally selecting transition matrices from past stress periods is a weak practice, as it may produce loss estimates that are not consistent with a given scenario and fails to recognize that conditions in the future may not precisely mirror conditions observed by the BHC in the past.

Sound rating transition models require two fundamental building blocks: a robust time series of data and well-calibrated, granular-risk rating systems. The Federal Reserve expects BHCs that use rating transition models to have robust time series of data that include a sufficient number of transitions, which allows BHCs to establish a statistically significant relationship between the transition behavior and macroeconomic variables. Data availability has been a widespread constraint inhibiting the development of granular transition models because a sufficient number of upgrades and downgrades are necessary to preclude sparse matrices. In order to overcome these data limitations, BHCs have often relied on third-party data to develop rating transition models. Consistent with the Federal Reserve’s general expectations, when using third-party data, BHCs should be able to demonstrate that the transition matrices estimated with external data are a reasonable proxy for the migration behavior of their portfolios. Rating transition models also require granular ratings systems that capture differences in the potential for defaults and losses for a given set of exposures in various economic environments. BHCs that lack well-calibrated, granular credit-risk rating systems are often unable to produce useful transition matrices.

BHCs with stronger practices typically had more granular ratings system and accounted for limitations in their data and/or credit rating systems by making adjustments to model assumptions or estimates, or by supplementing internal data with external data.

BHCs with weaker practices often failed to demonstrate that supplemented external data adequately reflected the ratings performance of the BHC’s portfolio. BHCs with weaker practices also sometimes relied on a risk rating process that historically resulted in lumpiness in rating upgrades and downgrades or material concentrations in one or two rating categories. As a result, these BHCs often produced transition matrices with limited sensitivity to scenario variables, and resulting estimates were more consistent with long-term average default rates than with default rates that would be experienced under severe economic stress.

Roll-Rate Models

Many BHCs have used roll-rate models to estimate losses for various retail portfolios. Roll-rate models generally estimate the rate at which loans that are current or delinquent in a given quarter roll into delinquent or default status in the next period. As a result, they are conceptually similar to rating transition models. The Federal Reserve expects BHCs that use roll-rate models to have a robust time series of data with sufficient granularity. The robust time series data allow the BHC to establish a strong relationship between roll rates and scenario variables, while the availability of granular data enables BHCs to model all relevant loan transitions and to segment the portfolio into subportfolios that exhibit meaningful variations in performance, particularly during the period of stress. In general, BHCs should estimate roll rates using models that are conditioned on scenario variables. For certain transition states where statistical relationships between roll rates and scenarios are weak (such as late stage loan delinquency), BHCs should incorporate conservative assumptions rather than relying solely on statistical relationships.

While roll-rate models have some advantages, including transparency and ease of use, they often have a weak predictive power outside the near future, particularly if they are not properly conditioned on scenario variables. As a result, some roll-rate models have limited usefulness for stress testing over a longer horizon, such as the nine-quarter planning horizon required in CCAR. Some BHCs have used roll-rate models in conjunction with other estimation approaches (such as a vintage model described below) that project losses for later periods. In general, it is a weaker practice to combine two different models, as it can introduce unexpected jumps in estimated losses over the planning horizon, though some BHCs have judgmentally weighed two different estimation methods to smooth projected losses. If BHCs combine two models, they should be able to demonstrate that such an approach is empirically warranted based on output analysis, including sensitivity analysis, and that the process of transitioning from one set of results to the other is consistent, well supported, and transparent.

Vintage Loss Models

Some BHCs use vintage loss models, also known as age-cohort-time models, to estimate losses for certain retail portfolios. BHCs that use vintage loss models generally segment their retail portfolios by vintage and collateral- or credit-quality-based segments. Losses are estimated using a multistep process—developing a baseline seasoning curve for each segment and using a regression model to estimate sensitivity of losses to macroeconomic variables at each seasoning level (e.g. four quarters after origination). This technique is commonly used in several vendor models, but BHCs also have developed and used proprietary models using this technique.

These models have several advantages (such as natural segmentation of portfolio by cohort and maturity) and ease of application to credit products (such as auto loans) that exhibit lifecycle effects. However, vintage models can be very challenging to construct, calibrate, and validate. In particular, it may be difficult to separately identify vintage effects from the effects of macroeconomic variables, which can result in poorly specified models. These models also assume that different cohorts will experience similar losses over time, generating results that are representative of average years, rather than during the period of stress. In using vintage models, it is important for a BHC to be able to demonstrate that the approach appropriately reflects its portfolio composition and history, and that modeled outputs are consistent with stressed conditions.

Charge-Off Models

A minority of BHCs have used net charge-off (NCO) models as either a primary loss-estimation model or a benchmark model. Typically, the NCO models BHCs used estimated a statistical relationship between charge-off rates and macroeconomic variables at a portfolio level, and often included autoregressive terms (lagged NCO rates). While some BHCs also incorporated variables that describe the underlying risk characteristics of the portfolio, NCO models that BHCs used for capital planning generally did not capture variation in sensitivities to risk drivers across important portfolio segments nor accounted for changes in portfolio risk characteristics over time. As a matter of general practice, BHCs should not use models that do not capture changes in portfolio risk characteristics over time and in scenarios used for stress testing as part of their internal capital planning.

NCO models often exhibit lower explanatory power than models that consider distinct portfolio risk drivers. In addition, NCO models implicitly assume that historical charge-off performance is a good predictor of future performance; however, the historical relationship between charge-offs and macro variables may not be realized under very stressful scenarios that fall outside the portfolio’s actual historical experience. Accordingly, a NCO model that is estimated without using sufficient segmentation or does not account for current or changing portfolio composition is unlikely to produce robust loss estimates. Thus, BHCs should avoid using such a NCO model as the primary loss-estimation approach for a material portfolio.

Scalar Adjustments

Some BHCs have used simple scalars to adjust portfolio loss estimate under a baseline scenario upward for stress scenarios. Scalars have been calibrated based on some combination of historical performance, the ratio of modeled stressed losses to baseline losses estimated for other portfolios, and expert judgment. Scalar adjustments are easy to develop, implement, and communicate; however, the approach has significant shortcomings, including lack of transparency and lack of sensitivity to changes in portfolio composition and scenario variables. Consequently, the use of these types of approaches should be, at most, limited to immaterial portfolios.

Available-for-Sale (AFS) and Held-to-Maturity (HTM) Securities

BHCs should test all credit-sensitive AFS and HTM securities for potential other-than-temporary impairment (OTTI) regardless of current impairment status. The threshold for determining OTTI for structured products should be based on cash-flow analysis and credit analysis of underlying obligors. Most BHCs used a ratings-based approach to determine OTTI of direct obligations such as corporate bonds, based on the projection of ratings migration under a stress scenario and a ratings-based OTTI threshold. However, some BHCs with weaker practice used a ratings-based approach that kept the ratings static over the scenario horizon.

BHCs should have quantitative methods that capture appropriate risk drivers and explicitly translate assumed scenario conditions into estimated losses. Estimation methods should generate results that conform to standard accounting treatment, are consistent with scenario conditions, and are appropriately sensitive to changes in key variables. Any assumptions (e.g. assumptions related to loss recognition) should be consistent with the intent of a stress testing exercise. Additionally, models should be independently validated for their use in projecting OTTI losses for specific classes of securities.

OTTI processes for AFS and HTM securities portfolios varied in sophistication across BHCs. BHCs with leading practices used estimation methods that capture both security-specific and country-specific performance data for relevant portfolios. For securitized products, they modeled the credit risk of underlying exposures (e.g. commercial real estate loans) to estimate potential losses. Where BHCs used management judgment, it was limited and well supported in the methodology documentation.

In addition, BHCs with leading practices chose conservative approaches and assumptions for OTTI loss estimation, such as recognizing losses in early quarters rather than over the entire scenario horizon. Though, under current accounting rules, OTTI losses are recognized only up to the amount of unrealized losses, some BHCs have taken a conservative approach to allow OTTI losses to exceed projected unrealized losses.

BHCs with lagging practices did not test all credit-sensitive securities for potential OTTI; rather, they tested only currently impaired positions or securities that met a certain criteria (e.g. only securities rated below investment grade) for OTTI. BHCs should not rely solely on a ratings-based threshold to determine OTTI for structured products. BHCs with lagging practices had OTTI loss-estimation methodologies that did not capture appropriate risk drivers or scenario conditions and/or were not applied at a sufficiently granular level. In some cases, BHCs excluded key explanatory variables for certain asset classes. For example, the unemployment rate was used to project OTTI losses for non-agency residential mortgage-backed securities (RMBS), but the housing price index (HPI) was excluded even though the theory and empirical evidence points to a strong relationship between mortgage losses and housing prices. As a result of these methodology deficiencies, these BHCs projected OTTI losses that were inconsistent with the risk characteristics of the portfolio and assumed scenario conditions.

Operational Risk

Best practices in operational-risk models are still evolving, and the Capital Plan Rule does not require BHCs to use advanced measurement approach (AMA) models for stressed operational-risk loss estimation. 34 However, BHCs that have developed a rich set of data to support the AMA should consider leveraging the same data and risk-management tools to estimate operational losses under a stress scenario, regardless of a particular methodology they choose to estimate losses.

Most operational-risk models use historical data on operational-risk loss events—incidences in which a BHC has experienced a loss or been exposed to loss due to inadequate or failed internal processes, people, or systems or from external events. Generally, operational-risk events are grouped into one of several event-type categories, such as internal fraud, external fraud, or damage to physical assets. 35 In general, BHCs should use internal operational-loss data as a starting point to provide historical perspective, and then incorporate forward-looking elements, idiosyncratic risks, and tail events to estimate losses. Most BHCs have supplemented their internal loss data with external data when modeling operational-risk loss estimates and scaled the losses to make the external loss data more commensurate with their individual risk profiles. The Federal Reserve expects such scaling approaches to be well supported. Few BHCs have incorporated business environment and internal control factors such as risk control self-assessments and other risk indicators into their operational-risk methodology. While the Federal Reserve does not expect BHCs to use these qualitative tools as direct inputs in a model, they can help identify areas of potential risk and help BHCs select appropriate scenarios that stress those risks.

Internal Data Collection and Data Quality

The Federal Reserve expects BHCs to have a robust and comprehensive internal data-collection method that captures key elements, such as critical dates (i.e. occurrence, discovery, and accounting), event types, and business lines. In general, BHCs should use complete data sets of internal losses when modeling, and not judgmentally exclude certain loss data.

Data quality and comprehensiveness have varied considerably across BHCs. BHCs with lagging practices often excluded certain internal loss data from model input for various reasons. Examples include

• excluding large items such as legal reserves and tax/compliance penalties;
• omitting losses from merged or acquired institutions mergers or acquisitions due to complications in collection and aggregation; and
• excluding loss data from discontinued business lines, even though the loss events were reasonably generic and applicable to remaining business lines within the organization.

Some BHCs have addressed observed outliers by omitting them from the data set, modeling them separately, or applying an add-on based on scenario analysis or management input. If BHCs do not have the data from potential mergers and acquisitions, one way to account for this limitation is to scale existing internal data using the size of operations and apply an add-on to applicable business lines or units of measure. If a BHC excludes data or uses data-smoothing techniques, especially as they affect large losses, it should have a well-supported rationale for doing so, and clearly document the rationale and the process. 36

The Federal Reserve expects BHCs to segment their loss data into units of measure that are granular enough to capture similar losses while balancing it with the availability of data. Most BHCs have segmented datasets by event type; however, some BHCs have segmented the loss data by consolidated business lines, event types, or some combination of the two.

Correlation with Macroeconomic Factors

Most BHCs have attempted to identify correlation between macroeconomic factors and operational-risk losses, but some have struggled to identify a clear relationship for some types of operational-risk loss events. BHCs that did not identify a significant correlation typically developed other methodologies, such as scenario analysis layered onto modeled results, to project stressed operational-risk losses. These approaches can be reasonable alternatives if BHCs can demonstrate that their approach results in sufficiently conservative loss estimates that are consistent with the stress scenario.

BHCs that identified correlations between macroeconomic factors and operational-risk elements typically had large data sets and often used external loss data to supplement internal data. These BHCs often identified correlations between loss frequency and macroeconomic factors for certain event types and adjusted the frequency distributions for the respective event type accordingly.

Common Operational-Loss-Estimation Approaches

Most BHCs have used their annual budgeting or forecasting process to estimate operational losses in the baseline scenario. The process typically uses a combination of historical loss data and management input at a business-line level. Some BHCs have used historical averages from internal loss data to estimate losses in the baseline scenario.

BHCs with stronger practices used a combination of approaches to incorporate historical loss experience, forward-looking elements, and idiosyncratic risks into their stressed loss projections. Using a combination of approaches can help address model and data limitations. Some BHCs used separate models for certain events types such as fraud or litigation, and used other approaches (e.g. using historical averages) for event types where no correlation with macroeconomic factors was identified. A simple approach may be acceptable depending on the size and complexity of the BHC as well as data and sophistication of models available to them. Very few BHCs have yet developed benchmarks to either challenge or further support the projections provided by their main models.

Regression Models

Most BHCs have used a regression model, either by itself or with another approach described below, to estimate operational-risk losses for stress scenarios. Some BHCs also have used a regression model for the baseline scenarios, albeit with different parameters. Operational-risk regression models are generally used to estimate two variables: loss frequency (i.e. the number of operational-risk losses) and loss severity (i.e. the loss amount).

BHCs that were able to identify significant correlation between macroeconomic variables and operational-risk losses have used regression models to stress the loss frequency or total operational-risk losses. Some macroeconomic variables were adjusted for the purpose of correlation analysis or to reflect time-lag assumptions. Most BHCs judgmentally chose time periods for estimation and model specification rather than justifying them with statistical evidence.

Most BHCs were not able to find meaningful correlation between macroeconomic variables and operational-risk loss severity. As a result, BHCs that used a regression model to estimate loss frequency typically applied the loss-severity assumption (e.g. static or four-quarter moving average) based on the most recent crisis period to estimate operational losses.

Modified Loss-Distribution Approach (LDA)

The LDA is an empirical modeling technique commonly used by BHCs subject to the AMA to estimate annual value-at-risk (VaR) measures for operational-risk losses based on loss data and fitted parametric distributions. The LDA involves estimating probability distributions for the frequency and the severity of operational loss events for each defined unit of measure, whether it is a business line, an event type, or some combination of the two. The estimated frequency and severity distributions are then combined, generally using a Monte Carlo simulation, to estimate the probability distribution for annual operational-risk losses at each unit of measure.

For purposes of CCAR, LDA models have generally been used in one of two ways: (1) by using a lower confidence interval than the 99.9th percentile used by the AMA, or (2) by adjusting the frequency based on outcomes of correlation analysis. BHCs that modified the LDA by using a lower confidence interval typically have used either the mean or median for the baseline estimates and higher confidence intervals—typically ranging from 70th percentile to 98th percentile—for the stressed estimates. Additionally, some BHCs have used different confidence intervals for different event types. The Federal Reserve does not require BHCs to use a particular percentile to produce stressed estimates. However, it expects BHCs to implement a credible, transparent process to select a percentile; be able to demonstrate why the percentile is an appropriate choice given the specific scenario under consideration; and perform sensitivity analyses around the selection of a percentile to test the impact of this assumption on model outputs. Some BHCs modified the LDA by adjusting frequency distributions based on the observed correlation between macroeconomic variables and operational-risk losses.

Scenario Analysis

Scenario analysis is a systematic process of obtaining opinions from business managers and risk-management experts to assess the likelihood and loss impact of plausible severe operational-loss events. Some BHCs have used this process to determine a management overlay that is added to losses estimated using a model-based approach. BHCs have used this overlay to incorporate idiosyncratic risks (particularly for event types where correlation was not identified) or to capture potential loss events that the BHC had not previously experienced. BHCs should be able to demonstrate the quantitative effect of the management overlay on final loss estimates.

Scenario analysis, if used effectively, can help compensate for data and model limitations, and allows BHCs to capture a wide range of risks, particularly where limited data are available. The Federal Reserve expects BHCs using scenario analysis to have a clearly defined process and provide an appropriate rationale for the specific scenarios included in their loss estimate. The process for choosing scenarios should be credible, transparent, and well supported.

Historical Averages

Some BHCs used historical averages of operational-risk losses, in combination with other approaches noted above, to estimate operational-risk losses under stress scenarios. For example, BHCs have used historical averages for event types where no correlation between macroeconomic factors and operational-risk losses was identified but used a regression model for event types where correlations were identified. A small number of BHCs have used historical averages as the sole approach to develop stressed loss estimates. When used alone, this approach is backward-looking and excludes potential risks the BHCs have not experienced. When using historical averages, BHCs should support the chosen time periods, thresholds, and any excluded or adjusted outliers and demonstrate that loss estimates are consistent with what are expected in the stress scenario.

Since legal exposure represents a significant portion of operational losses for many BHCs, a number of BHCs have analyzed and projected legal losses separately from non-legal losses. The Federal Reserve expects BHCs to include all legal reserves and settled legal losses in their total loss estimate for operational risk. BHCs have used various methods to estimate legal losses, such as applying a judgment-based add-on for significant losses; using legal reserves; using historical averages; or creating separate regression models for the clients, products, and business practices event type. To estimate litigation losses resulting from representations and warranties liabilities related to mortgage underwriting activities, some BHCs have developed hazard-rate models based on historical loan performance to estimate default rates and then estimated repurchase claim rates.

Market Risk and Counterparty Credit Risk

BHCs that have sizeable trading operations may incur significant losses from such operations under a stress scenario due to valuation changes stemming from credit and/or market risk, which may arise as a result of moves in risk factors such as interest rates, credit spreads, or equity and commodities prices, and counterparty credit risk owing to potential deterioration in the credit quality or outright default of a trading counterparty. 37 BHCs use different techniques for estimating such potential losses. These techniques can be broadly grouped into two approaches: probabilistic approaches that generate a distribution of potential portfolio-level profit/loss (P/L) and deterministic approaches that generate a point estimate of portfolio-level losses under a specific stress scenario.

Both approaches have different strengths and weaknesses. A probabilistic approach can provide useful insight into a range of scenarios that generate stress losses in ways that a deterministic stress testing approach may not be able to do. However, the probabilistic approach is complex and often lacks transparency, and as a result, it can be difficult to communicate the relevant scenarios to senior managers and the board of directors. In addition, the challenges inherent in tying probabilistic loss estimates to specific underlying scenarios can make it difficult for management and the board of directors to readily discern what actions could be taken to mitigate portfolio losses in a given scenario. Combined, these factors complicate the use of probabilistic approaches as the primary element in an active capital planning process that reflects well-informed decisions by senior management and the board of directors. The Federal Reserve expects BHCs using a probabilistic approach to provide evidence that such an approach can generate scenarios that are potentially more severe than what was historically experienced, and also to clearly explain how BHCs use the scenarios associated with tail losses to identify and address their idiosyncratic risks.

By comparison, a deterministic approach generally produces scenarios that are easier to communicate to senior management and the board of directors. However, a deterministic approach often uses a limited set of scenarios, and may miss certain scenarios that may result in large losses. The Federal Reserve expects BHCs using a deterministic approach to demonstrate that they have considered a range of scenarios that sufficiently stress their key exposures.

For CCAR, most BHCs generally relied on a deterministic approach. BHCs using deterministic approaches often relied on statistical models—for example, to inform the magnitude of risk-factor movements and covariances between risk factors—and also considered multiple scenarios as part of the broader internal stress testing supporting their capital planning process. BHCs using deterministic approaches used a three-step process to generate P/L losses under a stress scenario:

1. Design and selection of stress scenarios
2. Construction and implementation of the scenario (that is, translation to risk-factor moves)
3. Revaluation (and aggregation) of position and portfolio-level P&L under the stress scenarios

The Federal Reserve expects BHCs to have robust operational and implementation practices in all areas, including position inclusion, risk-factor representations, and revaluation methods.

Stress Scenarios

Most BHCs using deterministic approaches developed a set of broad narratives and considered a number of market shock scenarios that address the breadth of the BHCs’ risks before selecting the scenario included in their capital plans. In general, these BHCs used some combination of historical events and hypothetical projections to inform and develop the market shock scenarios. They also developed certain core themes or narratives for each scenario, which was sometimes supplemented with an overlay to capture additional nuances. BHCs generally developed the overlays using expert judgment based on the knowledge of their positions and market developments.

The Federal Reserve expects BHCs to consider multiple market shock scenarios as part of their internal stress testing. BHCs should develop and use stress scenarios that severely stress BHCs’ mark-to-market positions and account for BHCs’ idiosyncratic risks, in the event of a market-wide or firm-specific stress. In developing scenarios, BHCs should ensure that stress scenarios appropriately stress positions or products in which the BHC has a large market share (net or gross) or is a dominant player and should also consider more unusual basis risks arising from complex interlocking and interdependent positions, if such moves could result in large losses. BHCs that only use a scenario that closely mirrors the Federal Reserve’s global market shock component of the severely adverse and adverse scenarios should be aware that such an approach may omit significant risks that are unique to their positions, and that such omissions could lead to a negative assessment of a firm’s capital planning process. BHCs should clearly document the process they use to select stress scenarios, with sufficient justification and clear articulation of key aspects of the scenarios. 38

Translating Scenarios to Risk Factor Shocks

Once broad scenarios were developed, BHCs translated these scenarios into concrete specification of individual risk factors that were the actual inputs to pricing models, typically using the existing risk infrastructures and processes used for risk management, such as VaR and credit valuation adjustment (CVA). Most BHCs used instantaneous market shocks for stress testing, which assumed highly stressful outcomes that have typically occurred over a period of time (days, weeks, or months) will occur instantaneously. Given the uncertainty surrounding a firm’s ability to exit or manage positions during a period of severe market stress, this is an appropriate practice and suitably conservative for capital planning. Consistent with general supervisory expectations around risk-measurement processes, BHCs should clearly document the approximations and assumptions used as part of their measurement of risks under stress, assess the potential impacts, and address any deficiencies identified. 39

The size of shocks assumed in the stress scenario is often quite large. As a result, mechanical application of such shocks to current levels of risk factors could result in implausible outcomes such as negative risk-free rates or negative forward rates. BHCs should ensure that the proposed shocks produce results that are plausible. In particular, BHCs should take care in modeling dislocations and discordant moves of risk factors that normally move similarly. Additionally, while dislocations and discordant moves are expected under stress, BHCs should have a process to assess that the resulting joint moves of risk factors are reasonable. Also, the dislocations and discordant moves implied by a stress scenario may require risk-factor mappings that deviate from the normal mappings. BHCs should clearly document instances of such deviation and provide support. 40

Revaluation Methodologies and P/L Estimates

In principle, revaluation for stress testing can be carried out using the same infrastructure and calculators as conventional risk-measurement tools. However, practical revaluation methods may embed a number of approximations, which could introduce mismeasurement into the stress test results. In particular, VaR methodologies often use approximation methods for a number of reasons—for example, to economize on computational costs related to running a large number of scenarios daily. Although approximation methods may perform adequately for the risk-factor moves that are considered in normal conditions (for a small number of scenarios), BHCs should generally use full-revaluation methods for stress testing, given the very large risk-factor moves, especially for nonlinear positions with value dependent on multiple risk factors. BHCs can use approximation methods on a limited basis if extensive tests and analyses suggest that the potential mismeasurement from using such methods is not significant. BHCs should clearly support the process they use to ascertain the extent of such mismeasurements. Also, for certain parameters that are not easily market-observable and, therefore, cannot be inferred from traded instruments (e.g. correlations for credit-default baskets and correlations for certain interest-rate and exchange-rate pairs), BHCs should consider suitably perturbed values of the model parameters.

In addition, BHCs should ensure that P/L estimates under the stress scenario are relatively easy to interpret and explain. For example, BHCs with leading practices easily identified key P&L drivers in terms of positions, asset classes, and risk types. BHCs should also conduct sensitivity analysis to ensure that P/L estimates under the stress scenario are robust, without being unduly sensitive to small changes in inputs, assumptions, and modeling choices.

Counterparty and Issuer Defaults

Defaults of counterparties or issuers and/or reference entities are typically not embedded directly within the instantaneous market shock scenario. BHCs often use a model similar to that used for the incremental risk regulatory capital charge—a probabilistic approach based on some measure of PD, LGD, and EAD of counterparties or issuers—to estimate losses from possible defaults over some future horizon (e.g. to the typical margin period of risk). BHCs with leading practices also considered for their internal stress testing an explicit default scenario of one or more of their largest counterparties and/or customers. This approach has the benefit of allowing the BHC to consider targeted defaults of counterparties and customers to which the BHC has large exposures.

Risk Mitigants and Other Assumptions

Some BHCs have incorporated management responses to the stress, assuming, for example, some positions would be sold or hedged over time under the stress scenario. The Federal Reserve expects any assumptions about risk mitigation to be conservative. Where BHCs assume management actions that have the effect of reducing losses under the scenario, they should be able to demonstrate that such actions are consistent with established policy, supported by historical experience, and executable with high confidence in the market environment contemplated by the scenario. BHCs should recognize that their ability to take mitigating actions may be more limited in the stress scenario. For example, it may not be reasonable to assume that BHCs can easily sell their positions to other BHCs under the stress scenario. In addition, BHCs should avoid making unrealistic assumptions about their ability to foresee precisely how a scenario would play out, and take action on the basis of that information.