TCFD Support: Identifying Cheap Talk & Cherry Picking

By Julia Anna Bingler, Mathias Kraus, Markus Leippold 26 May, 2021

Bingler, Kraus & Leippold share how ClimateBert, their deep neural language model, exposes ineffectiveness in TCFD reporting

ClimateBert has assessed the amount, in total & differentiated by the 4 TCFD-categories, of climate-related info of >800 TCFD-supporting companies for a period of 6 yrs
Results show that TCFD seems to be cheap talk - climate risk reporting only increases slightly by 1.9%; substantial disclosure differences across sectors - energy & utilities most
Firms from North America & Europe are relatively aware of environ risks than Asia which only report half as much; mandatory disclosure & investors' growing awareness will help
Julia Anna Bingler
Author: Julia Anna Bingler
Julia Anna Bingler is a Doctoral Researcher at the Center of Economic Research at ETH Zurich. Her research focuses on climate transition risks and TCFD-related disclosures. She completed an M.Sc in Environmental Economics and Climate Change from London School of Economics (LSE) in 2015, and a second M.Sc in Economics from Leipzig University in 2017. Prior to joining ETH in 2019, she worked amongst others with the Wuppertal Institute, the German Institute for Economic Research (DIW Berlin) and Germanwatch on sustainable finance regulatory approaches and market-based climate policy. Besides her research, she regularly advises various stakeholders on national and international climate policy processes and participated in various UNFCCC climate conferences as an observer on finance-related topics.
Read more from Julia Anna Bingler →
Mathias Kraus
Author: Mathias Kraus
Mathias Kraus holds the junior professorship for Data Analytics at the Institute of Information Systems at FAU Nurnberg, Germany. He studied mathematics in the Bachelor’s program and computer science in the Master’s program at the Karlsruhe Institute of Technology (KIT) from 2010 to 2016. After completing his Master’s degree, he worked as a research assistant at the University of Freiburg in 2017 and as a PhD student at ETH Zurich from 2018 to 2020, where he successfully completed his PhD on “Deep Learning in Business Analytics: Methods and Applications” in October 2020. In the meantime, he spent a research stay at the University of Texas at Austin.
Read more from Mathias Kraus →
Markus Leippold
Author: Markus Leippold
Markus Leippold is a Professor at the Department of Banking at Finance of the University of Zurich and holds the Chair in Financial Engineering. From 2007 to 2009, he was an Associate Professor in Quantitative Finance at Imperial College London and served in the organization committee for the London Graduate School in Mathematical Finance. Until 2007, he was an Assistant Professor at the University of Zurich. In 2005, he was a visiting professor at the Federal Reserve Bank of New York. Before moving back to academia in 2002, he was working for Sungard, Trading and Risk Management Systems, and the Zurich Cantonal Bank. During his PhD studies, he was a research fellow at the Stern School of Business in New York. He obtained his PhD in financial economics from the University of St.Gallen, Switzerland, in 1999.
Read more from Markus Leippold →

TCFD (Task Force for Climate-related Financial Disclosures) is being championed by many as the golden standard since its launch in 2017 and widely referenced by governments and corporates. But is it truly effective? Have companies’ disclosures improved after the implementation? Or are they simply greenwashing themselves with TCFD? 

To answer this question, we sat down with Julia Anna Bingler, Mathias Kraus & Markus Leippold to find out how their deep neural language model ClimateBert evaluates corporates’ disclosures performances and identify the best & worst performers by regions and sectors. The full results of their research can be found in their recent paper Cheap Talk and Cherry-Picking: What ClimateBert has to say on Corporate Climate Risk Disclosures

CWR: Congratulations on your publication in SSRN. Before diving into the findings, can you share with us the goal of your research and why TCFD was picked as the framework to study?

Julia Anna Bingler (JAB), Mathias Kraus (MK) & Markus Leippold (ML): Thank you very much! The goal of our research is to get a closer look at climate risk disclosures of companies. Huge amounts of data about all kinds of topics (including climate risk) are made publicly available but it is tedious work to go through all this data and extract the relevant information.

For this task, we developed a language model — ClimateBERT — that is specialized on finding information about climate risk in corporate reports and documents. Such a model is beneficial for many parties, such as investors who can better assess whether their investments are prone to certain climate-related risks or not.

We used ClimateBERT to assess climate-related info of >800 TCFD-supporting co’s for 6 years…

It is also interesting for institutions that want to assess the effect of climate-related disclosure regulations. As a first step, we used ClimateBERT to assess the amount, in total and differentiated by the four TCFD-categories, of climate-related information of more than 800 TCFD-supporting companies for a period of six years. In addition, we are currently working on an extended version of our ClimateBert to enable better tracking of companies’ climate commitments and potential greenwashing over time.

…we focused on the TCFD because its their high relevance

We focused on the TCFD recommendations because of their high relevance. The TCFD categories have been perceived as the first disclosure framework on climate-related risks that could materially impact the business of the firm. In addition, the TCFD is referenced in various voluntary reporting frameworks like in the CDP questionnaire, and is being explicitly referenced for mandatory disclosure regulation in jurisdictions like New Zealand, the UK and the EU and Switzerland forthcoming.

CWR: To give our readers a better understanding on your research, can you walk us through your methodology and any limitations? 

JAB, MK & ML: ClimateBERT builds on the latest development in the natural language processing field. Instead of working with pre-defined keywords or simply mapping text to a set of unordered words, we use contextual language models that can put terms within the text into the context of other terms in the text. We trained it on thousands of human-labeled sentences with climate risk content from company reports. ClimateBert is able to detect, extract and classify climate-related disclosures in company filings, differentiated by the four TCFD categories governance, strategy, risk management, and metrics and targets.

ClimateBERT 2.0 will focus on climate commitments & specificity…

At first, we also wanted to train the algorithm to differentiate between physical and transition risk disclosures. But this would have required a lot of additional human-labeled data. Given our initial research interest – to assess whether voluntary commitments to the TCFD result in higher levels of climate risk disclosures – we decided to focus as a first step on the four TCFD categories. But we are currently training ClimateBERT 2.0 on climate commitments and specificity – and there might be a ClimateBERT 3.0 in the future to differentiate between physical and transition risks.

CWR: What were some of the key findings and any surprises from the research? 

JAB, MK & ML: On a methodological level, we were quite surprised, how well the current state-of-the-art language models perform on new, and arguably difficult, language tasks.

It’s cheap talk… the amount of climate risk reporting increases only ~1.9% from 2017-2020

In terms of TCFD disclosures, we were surprised that supporting the TCFD seems to be cheap talk: By applying ClimateBERT on the annual reports of more than 800 TCFD-supporting companies, we find that the average amount of climate risk reporting increases only slightly by approximately 1.9 percentage points after the launch of the recommendations in 2017 until 2020.

Second, we find some evidence for cherry picking, i.e. that most of this overall increase is due to increased governance and risk management disclosure after the launch of the TCFD recommendations in 2017. However, there is hardly any increase in strategy and metrics and targets which most likely provides the most essential and material information to stakeholders.

CWR: Can you walk us through the best & worst performing sectors? 

JAB, MK & ML: We find substantial differences in climate disclosures when we differentiate by sectors. Energy and utilities disclose the highest amount of climate-related information, followed by transportation and financials. Materials, industrials, and real estate exhibit lower than average disclosure levels for 2014 until 2020. In the energy sector and utilities, the relevant greenhouse gas emissions are relatively straightforward to track (so-called Scope 1 emissions). Hence, gathering this information is associated with fewer information costs and became a common reporting metric in most energy-intensive sectors, also before the arrival of the TCFD recommendations.

CWR: Differences in performance can also be observed from a regional perspective – who is doing the best or worst? Why do you think this is the case and how can we improve the worst performers? 

JAB, MK & ML: In terms of regional differentiation, we find that TCFD supporting firms headquartered in North America and Europe disclose relatively more information than firms from Asia, which roughly report only half as much as their North American counterparts.

Firms from North America & Europe disclose more, likely because they are relatively aware of environmental risks

A reason for this might be that TCFD supporting firms from North America and Europe are relatively aware of environmental risks. Also, firms in Europe might have prepared earlier, due to the NFRD regulation that was introduced in 2014. To further explore differences in the preparedness of European, US, and Asian companies would be an exciting avenue for future research

We think that an improvement can be achieved through two ways. One way could be by translating voluntary climate risk disclosures into regulatory disclosures. Another possibility that likely will happen in the future is that stakeholders become more aware of the need for climate risk disclosures and investors will put a risk premium on investments into firms that do not disclose sufficiently.

CWR: Climate-related financial risks can be split into physical (acute & chronic) and carbon transition. Bundling them together can lead to risks being hidden, e.g. companies/sectors could be doing more on transition risks but little on physical risks and so still be exposed. Will your next iteration of this research separate these risks? 

JAB, MK & ML: Indeed, we are currently working on ClimateBERT 2.0 which will not only classify paragraphs into one of the TCFD categories, but which will also start assessing the content of the paragraphs on multiple dimensions. One of these dimensions is the specificity by which the firms make statements. Independent of the risk category (physical or transition), the statements will be assessed by their quality. And, as already stated before, there might be a ClimateBERT 3.0 in the future to differentiate between physical and transition risks.

Further Reading

More on Latest

  • Water Risk In The ICT Sector – Analysing >3,000 ICT sector locations, AWS’s Sarah Wade shares where the water risks are, which type of water risks they are facing & what the sector can do to become a water stewardship leader
  • 3 Reasons Why APAC Banks Must De-risk Now – CWR’s Dharisha Mirando explains why APAC banks must lead & de-risk, plus she catches you up on the latest new stats from science and finance, so you won’t be blindsided
  • How Water Risks Will Impact Credit Ratings – Water risks are relevant for a wide variety of sectors & asset classes yet, incorporating them in credit rating decisions is difficult. Fitch Ratings’ Justin Sloggett shares what to do
  • A Wave Of Change: Companies’ Role In Building A Water-secure World – Despite the pandemic, companies disclosing to CDP are up 20%. Their Laureen Missaire shares the latest trends from their 2020 global water report
  • Adapting To Climate Change – Like it or not, climate impacts are getting increasingly ‘up close & personal’ yet, adaptation finance is lagging. Banks & investors should get on top of it as it makes business sense, suggest BNP’s Chaoni Huang & Jonathan Ho