TCFD Support: Identifying Cheap Talk & Cherry Picking

By Julia Anna Bingler, Mathias Kraus, Markus Leippold 26 May, 2021

Bingler, Kraus & Leippold share how ClimateBert, their deep neural language model, exposes ineffectiveness in TCFD reporting

TCFD (Task Force for Climate-related Financial Disclosures) is being championed by many as the golden standard since its launch in 2017 and widely referenced by governments and corporates. But is it truly effective? Have companies’ disclosures improved after the implementation? Or are they simply greenwashing themselves with TCFD? 

To answer this question, we sat down with Julia Anna Bingler, Mathias Kraus & Markus Leippold to find out how their deep neural language model ClimateBert evaluates corporates’ disclosures performances and identify the best & worst performers by regions and sectors. The full results of their research can be found in their recent paper Cheap Talk and Cherry-Picking: What ClimateBert has to say on Corporate Climate Risk Disclosures. 

CWR: Congratulations on your publication in SSRN. Before diving into the findings, can you share with us the goal of your research and why TCFD was picked as the framework to study?

Julia Anna Bingler (JAB), Mathias Kraus (MK) & Markus Leippold (ML): Thank you very much! The goal of our research is to get a closer look at climate risk disclosures of companies. Huge amounts of data about all kinds of topics (including climate risk) are made publicly available but it is tedious work to go through all this data and extract the relevant information.

For this task, we developed a language model — ClimateBERT — that is specialized on finding information about climate risk in corporate reports and documents. Such a model is beneficial for many parties, such as investors who can better assess whether their investments are prone to certain climate-related risks or not.

We used ClimateBERT to assess climate-related info of >800 TCFD-supporting co’s for 6 years…

It is also interesting for institutions that want to assess the effect of climate-related disclosure regulations. As a first step, we used ClimateBERT to assess the amount, in total and differentiated by the four TCFD-categories, of climate-related information of more than 800 TCFD-supporting companies for a period of six years. In addition, we are currently working on an extended version of our ClimateBert to enable better tracking of companies’ climate commitments and potential greenwashing over time.

…we focused on the TCFD because its their high relevance

We focused on the TCFD recommendations because of their high relevance. The TCFD categories have been perceived as the first disclosure framework on climate-related risks that could materially impact the business of the firm. In addition, the TCFD is referenced in various voluntary reporting frameworks like in the CDP questionnaire, and is being explicitly referenced for mandatory disclosure regulation in jurisdictions like New Zealand, the UK and the EU and Switzerland forthcoming.

CWR: To give our readers a better understanding on your research, can you walk us through your methodology and any limitations? 

JAB, MK & ML: ClimateBERT builds on the latest development in the natural language processing field. Instead of working with pre-defined keywords or simply mapping text to a set of unordered words, we use contextual language models that can put terms within the text into the context of other terms in the text. We trained it on thousands of human-labeled sentences with climate risk content from company reports. ClimateBert is able to detect, extract and classify climate-related disclosures in company filings, differentiated by the four TCFD categories governance, strategy, risk management, and metrics and targets.

ClimateBERT 2.0 will focus on climate commitments & specificity…

At first, we also wanted to train the algorithm to differentiate between physical and transition risk disclosures. But this would have required a lot of additional human-labeled data. Given our initial research interest – to assess whether voluntary commitments to the TCFD result in higher levels of climate risk disclosures – we decided to focus as a first step on the four TCFD categories. But we are currently training ClimateBERT 2.0 on climate commitments and specificity – and there might be a ClimateBERT 3.0 in the future to differentiate between physical and transition risks.

CWR: What were some of the key findings and any surprises from the research? 

JAB, MK & ML: On a methodological level, we were quite surprised, how well the current state-of-the-art language models perform on new, and arguably difficult, language tasks.

It’s cheap talk… the amount of climate risk reporting increases only ~1.9% from 2017-2020

In terms of TCFD disclosures, we were surprised that supporting the TCFD seems to be cheap talk: By applying ClimateBERT on the annual reports of more than 800 TCFD-supporting companies, we find that the average amount of climate risk reporting increases only slightly by approximately 1.9 percentage points after the launch of the recommendations in 2017 until 2020.

Second, we find some evidence for cherry picking, i.e. that most of this overall increase is due to increased governance and risk management disclosure after the launch of the TCFD recommendations in 2017. However, there is hardly any increase in strategy and metrics and targets which most likely provides the most essential and material information to stakeholders.

CWR: Can you walk us through the best & worst performing sectors? 

JAB, MK & ML: We find substantial differences in climate disclosures when we differentiate by sectors. Energy and utilities disclose the highest amount of climate-related information, followed by transportation and financials. Materials, industrials, and real estate exhibit lower than average disclosure levels for 2014 until 2020. In the energy sector and utilities, the relevant greenhouse gas emissions are relatively straightforward to track (so-called Scope 1 emissions). Hence, gathering this information is associated with fewer information costs and became a common reporting metric in most energy-intensive sectors, also before the arrival of the TCFD recommendations.

CWR: Differences in performance can also be observed from a regional perspective – who is doing the best or worst? Why do you think this is the case and how can we improve the worst performers? 

JAB, MK & ML: In terms of regional differentiation, we find that TCFD supporting firms headquartered in North America and Europe disclose relatively more information than firms from Asia, which roughly report only half as much as their North American counterparts.

Firms from North America & Europe disclose more, likely because they are relatively aware of environmental risks

A reason for this might be that TCFD supporting firms from North America and Europe are relatively aware of environmental risks. Also, firms in Europe might have prepared earlier, due to the NFRD regulation that was introduced in 2014. To further explore differences in the preparedness of European, US, and Asian companies would be an exciting avenue for future research

We think that an improvement can be achieved through two ways. One way could be by translating voluntary climate risk disclosures into regulatory disclosures. Another possibility that likely will happen in the future is that stakeholders become more aware of the need for climate risk disclosures and investors will put a risk premium on investments into firms that do not disclose sufficiently.

CWR: Climate-related financial risks can be split into physical (acute & chronic) and carbon transition. Bundling them together can lead to risks being hidden, e.g. companies/sectors could be doing more on transition risks but little on physical risks and so still be exposed. Will your next iteration of this research separate these risks? 

JAB, MK & ML: Indeed, we are currently working on ClimateBERT 2.0 which will not only classify paragraphs into one of the TCFD categories, but which will also start assessing the content of the paragraphs on multiple dimensions. One of these dimensions is the specificity by which the firms make statements. Independent of the risk category (physical or transition), the statements will be assessed by their quality. And, as already stated before, there might be a ClimateBERT 3.0 in the future to differentiate between physical and transition risks.

Further Reading

• Wipeout – 8 Reasons Why Stress Testing For Chronic Risks Will Strand Assets – Systemic shocks from water risks are inevitable unless action is taken to reduce emissions. CWR’s Dharisha Mirando & Debra Tan share 8 things you need to know about stress testing your portfolio
• The CWR Survival Guides to Avoiding Atlantis – Sea levels can be 3m by 2100, putting urban real estate equivalent to 22 Singapores underwater in just 20 APAC capitals & cities. With US$5.7trn of annual GDP at stake, get on top of the new risk landscape to survive
• Business of Fashion’s Inaugural Sustainability Index: The Sustainability Gap – BoF’s Sarah Kent who led the inaugural sustainability index talks to us on the why, how & what’s next. Plus, see how the largest 15 global fashion companies scored
• The Adaptation Principles: 6 Ways to Build Resilience to Climate Change – Adaptation cannot be an afterthought to development as climate change will impact the macroeconomic situation. World Bank’s Dr Stephane Hallegatte, Dr Jun Rentschler & Dr Julie Rozenberg share 6 principles

More on Latest

• Water Risk In The ICT Sector – Analysing >3,000 ICT sector locations, AWS’s Sarah Wade shares where the water risks are, which type of water risks they are facing & what the sector can do to become a water stewardship leader
• 3 Reasons Why APAC Banks Must De-risk Now – CWR’s Dharisha Mirando explains why APAC banks must lead & de-risk, plus she catches you up on the latest new stats from science and finance, so you won’t be blindsided
• How Water Risks Will Impact Credit Ratings – Water risks are relevant for a wide variety of sectors & asset classes yet, incorporating them in credit rating decisions is difficult. Fitch Ratings’ Justin Sloggett shares what to do
• A Wave Of Change: Companies’ Role In Building A Water-secure World – Despite the pandemic, companies disclosing to CDP are up 20%. Their Laureen Missaire shares the latest trends from their 2020 global water report
• Adapting To Climate Change – Like it or not, climate impacts are getting increasingly ‘up close & personal’ yet, adaptation finance is lagging. Banks & investors should get on top of it as it makes business sense, suggest BNP’s Chaoni Huang & Jonathan Ho