Energy Trust of Oregon
Reduce budget for statistical methods. We don’t get much out of RTF work on statistical methods for evaluation. I don’t know if others do. I think there is a fantasy that RTF can bless methods and regulators can have more confidence in evaluation. I believe that aside from massive projects like LBNL’s multiyear work on validating hourly analysis tools for commercial buildings, which are beyond RTF’s scope and resources, evaluation tends to be so situation-dependent that you can’t by a method and proceed with impunity; it takes expertise to craft the method to the situation, and to review that application. So I’m less convinced that this is constructive. I think evaluation methods will evolve through the work of others.
- Response: This is still a priority area for the RTF to address questions that arise about statistical methods. Proposed budget was dropped from assumed 25% to 10% of two contract analysts time, with the assumption that more of this work will focus on reviewing efforts of others.
Add budget for capacity benefits and load shape analysis. There is some work that I think deserves mention here. RTF is wrapping up a study to prioritize load shapes for improvement. There will be work to influence the regional metering study based on this. Also, Energy Trust plans to review summer peak savings estimates from our ten highest priority RTF load shapes for reasonableness and may propose some near-term enhancements to assure that we are doing good planning while we wait several years for the new wave of metered data to arrive and be synthesized for us. We hope RTF will both help on the review board for this project, and then take a step away to assess and if necessary correct our work. If it looks productive, RTF may take on the next ten.
- Response: Additional budget was added to the contract analysts for engagement in load profile research, which is meant to be inclusive of activities related to improving our load profiles. It was also clarified that the ProCost updates include a focus on enhancing the estimation of capacity benefits.
Correct typo. Under Existing Measure Updates, the spreadsheet detail tab says “26 UES slated to sunset, 3 assumed to be address in 2016, 9 are small saver, 9 planning which might involve minimal update, assumes contract analysts do all the work” – not sure what the highlight is supposed to be, but I’m guessing it’s not 2016. 2017?
- Response: Fixed.
Applied Energy Group Comments
It is unclear whether sufficient funds are allocated for measure/protocol development. Overall costs in the new measure development category have gone down relative to the 2017 work plan, in spite of targeting a comparable number/mix of new measures. It would be really helpful to note throughout where cost assumptions have changed significantly year-on-year and why.
- Response: Generally, the amount per unit was similar or slightly higher for 2018 compared to the final 2017 work plan. The one category of note was the new standard protocols. In the draft proposal, those were scoped to take the same resource as existing; however, this has been updated to more correctly reflect that additional resources are needed for the development. This is now more in line with 2017. One note, the the amount of contract RFP support for measure development was cut in the proposed work plan to account for added contract RFP to support 8th plan development.
The RTF would benefit from a more systematic, proactive process of scoping new measures, better targeting new measure funds, and measuring success. Since there is a significant amount of past/active market transformation and emerging technology activity in the region, some of the allocations in the “Regional Coordination” and “RTF Management” categories may already be earmarked for developing such processes and developing this data via research and market analysis. However, if not, some budget allocation should be considered for “Ongoing New Measure Scoping/Solicitation,” whether as additions to pre-existing line items or as a standalone line item.
- Response: The RTF is undertaking some proactive scoping for whole building measures in 2017. There is also some proposed allocation to scoping for small/rural. At this time, the work plan does not propose any additional proactive scoping, but rather plan to continue to rely on the region for bringing forward new measure proposals.
Tasks such as “Statistical Methods Support” and the new measure development for whole-building performance should account for the increasing accessibility of high-frequency AMI data and other non-traditional data sources/streams and their cross-cutting potential for both improving ex ante planning rigor and displacing traditional evaluation methods. (The use of this kind of data is variously referred to as “M&V 2.0” and “EM&V 2.0.”)
- Response: This comment is noted and will be considered as part of those efforts.