Let’s say that you have an international clinical trial that shows a new drug (SuperDrug) perform better than the previous standard of care (OldDrug). Also assume that individuals with a specific comorbidity–let’s call it EF–respond less well to the SuperDrug treatment. If you live in a country where comorbidity EF is common, how well do you think SuperDrug will work in your population?

This is the question posed by Turner et al. (2023) in their recent PharmacoEconomics paper. The general problem country decisionmakers face is the following:

When study populations are not randomly selected from a target population, external validity is more uncertain and it is possible that distributions of effect modifiers (characteristics that predict variation in treatment effects) differ between the trial sample and target population

Many of you may have guessed that my comorbidity EF actually stands for an effect modifier. Four classes of effect modifiers the authors consider include:

• Patient/disease characteristics (e.g. biomarker prevalence),
• Setting (e.g. location of and access to care),
• Treatment (e.g. timing, dosage, comparator therapies, concomitant medications)
• Outcomes (e.g. follow-up or
• timing of measurements)

See Beal et al. (2022) for a potential checklist for effect modifiers.

In their paper, the authors examine the problem of transportability. What is transportability?

Whereas generalisability relates to whether inferences from a study can be extended to a target population from which the study dataset was sampled, transportability relates to whether
inferences can be extended to a separate (external) population from which the study sample was not derived.

Key cross-country differences that may make transportability problematic include effect modifiers
such as disease characteristics, comparator therapies and treatment settings.

What is the problem of interest:

Typically, decision makers are interested in the target population average treatment effect (PATE): the average effect of treatment if all individuals in the target population were assigned the treatment. However, researchers commonly have access only to a sample and must estimate the study sample average treatment effect (SATE).

Key assumptions to estimate PATE are included below:

Primarily, there are two key items to address (for RCTs at least): (i) are there differences in the distributions of characteristics between study and population of the target country/geography and (ii) are these characteristics effect modifiers [or for single arm trials with external controls, prognostic factors].

One can test for differences in the distribution of covariates using mean differences of propensity scores, examining propensity score distributions, as well formal diagnostic tests to identify the absence of an overlap. Univariate standardized mean differences (and relevant tests) can subsequently be used to examine drivers of overall differences. If only aggregate data are available, one may be limited to comparing differences in mean values.

To test if a variable is an effect modifier, the authors recommend the following approaches:

Parametric models with treatment-covariate interactions can be used to detect effect modification. Where small study samples result in power issues or where unknown functional
forms increase the risk of model misspecification, machine learning techniques such as Bayesian additive regression trees could be considered, and the use of directed acyclic
graphs may be particularly crucial for selecting effect modifiers in this case.

Approaches for adjusting for effect modifiers vary depend on whether a research has access to individual patient data.

• With IPD: Use outcome regression-based methods, matching, stratification, inverse odds of participation weighting and doubly robust methods combining matching/weighting with regression adjustment.
• Without IPD. Use population-adjusted indirect treatment comparisons (e.g., matching-adjusted indirect comparisons).

To determine which in-country data–typically real-world data–should be used as the target population, one could consider a variety of tools such as EUnetHTA’s REQueST or the Data Suitability Assessment
Tool (DataSAT) tool from NICE.

You can read more recommendations on how to best validate transportability issues in the full paper here.

Read More

Imagine the body’s cells are well-behaved students in the classroom. The “teachers” are tumor suppressor genes, and they make sure cells follow the rules. But when tumor suppressor genes are away, cells may go astray.

Credit: Virginia Tech

Read More

AI developers are taking revenue, data and users away from news publications by building competing products, the News Media Alliance claims.

Artificial intelligence developers heavily rely on illegally scraping copyrighted material from news publications and journalists to train their models, a news industry group has claimed.

On Oct. 30, the News Media Alliance (NMA) published a 77-page white paper and accompanying submission to the United States Copyright Office that claims the data sets that train AI models use significantly more news publisher content compared to other sources.

As a result, the generations from AI “copy and use publisher content in their outputs” which infringes on their copyright and puts news outlets in competition with AI models.

“Many generative AI developers have chosen to scrape publisher content without permission and use it for model training and in real-time to create competing products,” NMA stressed in an Oct. 31 statement.

On Monday, the News/Media Alliance published a White Paper and a technical analysis and submitted comments to the @CopyrightOffice on the use of publisher content to power generative artificial intelligence technologies (#GAI). https://t.co/Zr05e7nZTS

— News/Media Alliance (@newsalliance) October 31, 2023

The group argues while news publishers make investments and take on risks, AI developers are the ones rewarded “in terms of users, data, brand creation, and advertising dollars.”

Reduced revenues, employment opportunities and tarnished relationships with its viewers are other setbacks publishers face, the NMA noted its submission to the Copyright Office.

To combat the issues, the NMA recommended the Copyright Office declare that using a publication’s content to monetize AI systems harms publishers. The group also called for various licensing models and transparency measures to restrict the ingestion of copyrighted materials.

The NMA also recommends the Copyright Office adopt measures to scrap protected content from third-party websites.

The Guardian has accused Microsoft of damaging its journalistic reputation by publishing an AI-generated poll speculating on the cause of a woman’s death next to an article by the news publisher. https://t.co/tOie87HSyA

— News/Media Alliance (@newsalliance) November 1, 2023

The NMA acknowledged the benefits of generative AI and noted that publications and journalists can use AI for proofreading, idea generation and search engine optimization.

OpenAI’s ChatGPT, Google’s Bard and Anthropic’s Claude are three AI chatbots that have seen increased use over the last 12 months. However, the methods to train these AI models have been criticized, with all facing copyright infringement claims in court.

Related: How Google’s AI legal protections can change art and copyright protections

Comedian Sarah Silverman sued OpenAI and Meta in July claiming the two firms used her copyrighted work to train their AI systems without permission.

OpenAI and Google were hit with separate class-action suits over claims they scraped private user information from the internet.

Google has said it will assume legal responsibility if its customers are alleged to have infringed copyright for using its generative AI products on Google Cloud and Workspace.

“If you are challenged on copyright grounds, we will assume responsibility for the potential legal risks involved.

However, Google’s Bard search tool isn't covered by its legal protection promise.

OpenAI and Google did not immediately respond to a request for comment.

Magazine: AI Eye: Real uses for AI in crypto, Google’s GPT-4 rival, AI edge for bad employees

Read More