[ Research & Publications ]

Research & Publications

Applied machine learning research in ambient smart-home activity recognition, self-supervised representation learning, and multimodal fusion at the University of Georgia.

[ Academic Research ]

Thesis

M.S. Thesis · University of Georgia, 2026Complete & Defended

Reading smart-home sensors four ways

Multimodal Self-Supervised Activity Recognition on CASAS Smart-Home Sensor Streams

A smart home can recognise what a resident is doing without cameras, just from ambient motion and door sensors clicking on and off. My thesis investigates how best to read those sparse event logs, finding that the optimal representation depends directly on the home's topology.

Four Synchronized Views of Sensor Streams
SequenceTransformer

Ordered sensor timestamps, states, cyclic time-of-day, and inter-event intervals.

Text SummaryNLP Encoder

Deterministic natural language narrative summarizing room visits and movement pace.

Raster ImageCNN

96x96 spatial activation heatmap projected directly onto the home floor plan.

Graph NetworkGNN

Spatial-temporal event and sensor topology with local message passing and global attention.

Download Thesis (PDF)

58 pages · 2.3 MB · Advisor: Dr. Fei Dou

4

CASAS Homes

Aruba, Cairo, Kyoto and Milan

4

Synchronized Views

Sequence, text, raster image & graph

30

Events Per Window

Stride of 15 with balanced overlap

3

Random Seeds

Evaluated across seeds 10, 30 and 50

Benchmark ComparisonMean over 3 seeds

Aruba House

Graph + Image + Sequence

0.850 W-F1

vs SICA (0.847)

Cairo House

Graph + Text + Image

0.842 W-F1

vs SICA (0.755)

Kyoto House

Image + Text + Sequence

0.843 W-F1

vs SICA (0.821)

[ Conference Proceedings ]

Submitted Paper

Submitted to IEEE PerCom 2027Under Review · IEEE PerCom 2027

FACET

FACET: Factorized Ambient-Sensor Experts with Local Competence-Weighted Fusion for Smart-Home Activity Recognition

A machine learning paper on recognizing daily human activities from ambient smart-home sensors using factorized expert models and local competence-weighted fusion.

Joint research in Fei Dou's research lab at the University of Georgia. Submitted to IEEE PerCom 2027 (IEEE International Conference on Pervasive Computing and Communications).