> 교육용 가상 원고입니다. 의도적인 인용·숫자·보고 오류가 있습니다. 실제 연구 결과나 임상 근거로 사용하지 마십시오.

# Efficacy of LLM-assisted counseling in type 2 diabetes: a randomised controlled trial

## Abstract

**Background:** Digital tools may support diabetes self-management, and large language models (LLMs) now make richer counseling possible. **Methods:** We randomized adults with type 2 diabetes to an LLM counseling chatbot or a control arm and measured glycaemic control at 8 weeks. **Results:** Of 220 randomized participants, 198 completed follow-up. Mean HbA1c reduction was greater in the chatbot arm (−0.6% vs −0.2%, p = 0.01). **Conclusions:** LLM-assisted counseling is effective for glycaemic control in type 2 diabetes.

## Introduction

Type 2 diabetes management depends on sustained self-care. Intensive glycaemic control reduces microvascular complications [1, 2], and people with diabetes had worse outcomes during the COVID-19 pandemic [3]. Conversational agents have shown promise for patient education [4, 5], and large language models (LLMs) now make richer counseling possible [6]. A recent trial of chatbot-delivered counseling reported improved adherence [7]. We evaluated whether an LLM counseling chatbot improves glycaemic control.

## Methods

Adults with type-2 diabetes and a smartphone were enrolled in primary care and randomized 1:1. The intervention arm received an LLM counseling chatbot; a control arm received standard printed diabetes education materials. The primary outcome was change in HbA1c, assessed at 8 weeks. Secondary outcomes included medication adherence and treatment satisfaction. Analyses followed an intention-to-treat (ITT) principle using a linear mixed model. A p value < 0.05 was considered significant.

The study received institutional review board (IRB) approval and all participants provided written informed consent.

## Results

Of 220 randomized participants, 198 completed follow-up. Baseline characteristics are shown in Table 1. Mean HbA1c reduction was greater in the chatbot arm at 8 weeks (−0.6% vs −0.2%, p=0.01). Medication adherence and treatment satisfaction also favoured the intervention. As shown in Table 2, 45 participants (42.0%) in the chatbot arm reached the HbA1c target, compared with 25 (26.0%) in the control arm. Figure 2 shows the change in HbA1c over time.

Table 1. Baseline characteristics of randomized participants.

| Characteristic | Chatbot (n = 110) | Control (n = 110) | Total (n = 220) |
|---|---|---|---|
| Age, years, mean (SD) | 58.2 (9.1) | 57.6 (9.8) | 57.9 (9.4) |
| Female, n (%) | 52 (47.3) | 49 (44.5) | 101 (45.9) |
| Insulin use, n (%) | 31 (28.2) | 27 (24.5) | 58 (26.4) |
| Education, n (%) | | | |
| Primary | 20 (18.2) | 24 (21.8) | 44 (20.0) |
| Secondary | 55 (50.0) | 52 (47.3) | 107 (48.6) |
| Tertiary | 35 (31.8) | 34 (30.9) | 69 (31.4) |

Table 2. HbA1c target attainment at 8 weeks.

| Outcome | Chatbot (n = 100) | Control (n = 98) | Total |
|---|---|---|---|
| Reached HbA1c < 7% | 45 (45.0) | 25 (25.5) | 70 (35.4) |
| Did not reach | 55 (55.0) | 72 (74.5) | 128 (64.6) |
| Total | 100 | 98 | 198 |

Adverse events were rare: 3.0% reported anxiety about the chatbot's advice, 6.1% reported technical difficulties, and 92.9% reported no problems.

## Discussion

LLM-assisted counseling improved glycaemic control and was well received. Mean HbA1c reduction was greater in the chatbot arm at 8 weeks (−0.6% vs −0.2%, p = 0.01). These findings show that LLM counseling is effective and should be adopted in routine diabetes care. It should be noted that in order to generalise these results, longer follow up is needed.

Our results are consistent with prior work on conversational agents [4, 5] and with patient acceptability studies [9]. Intensive glycemic control reduces microvascular complications [1], which may possibly suggest that the observed HbA1c change could potentially translate into clinical benefit.

### Limitations

Participants were unblinded, follow-up was short, and the trial was conducted at a single centre.

## Declarations

**Funding:** Supported by an institutional research grant.
**Competing interests:** The authors declare no competing interests.
**Data availability:** De-identified data are available from the corresponding author on reasonable request.

## References

1. UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet. 1998;352(9131):837-853. doi:10.1016/S0140-6736(98)07019-6
2. The Diabetes Control and Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1994;329(14):977-986. doi:10.1056/NEJM199309303291401
3. Mehra MR, Desai SS, Kuy S, Henry TD, Patel AN. Cardiovascular disease, drug therapy, and mortality in Covid-19. N Engl J Med. 2020;382(25):e102. doi:10.1056/NEJMoa2007621
4. Laranjo L, Dunn AG, Tong HL, et al. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc. 2018;25(9):1248-1258. doi:10.1093/jamia/ocy072
5. Laranjo L, Dunn AG, Tong HL, et al. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc. 2018;25(9):1248-1258.
6. Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. 2023;29(8):1930-1940. doi:10.1038/s41591-023-02448-8
7. Park JH, Lee SY, Kim MJ. Chatbot-delivered diabetes counseling improves medication adherence: a randomized trial. JMIR Diabetes. 2023;8(2):e47812. doi:10.2196/47812
8. Nadarzynski T, Miles O, Cowie A, Ridge D. Acceptability of artificial intelligence (AI)-led chatbot services in healthcare: a mixed-methods study. Digit Health. 2019;5:2055207619871808. doi:10.1177/2055207619871808