News

2023.04, "LemurDx" is accepted (with minor revision) to IMWUT.
2023.04, "AI for human assessment" received Honorable Case Study Recognition (top 3) at ACM CHI 2023.
2023.04, "IMUPoser" received Honorable Mention Award (top 5%) at ACM CHI 2023.
2023.01, Following on from last year, I'll be serving as one of the Program Committee Members at ACM CHI LBW 2023.
2023.01, Attended APSARD 2023 in Orlando. First time in a medical conference.
2023.01, Super lucky to get three papers accepted to CHI2023. Awesome collaborators!
2022.12, I received Snap Research Fellowship!
2022.11, "AI for human assessment" is accepted to CHI2023 case study.
2022.10, "PrISM-Tracker" is accepted (with minor revision) to IMWUT.

2022.09, Passed CommTalk at CMU HCII, great chance for practice with lots of useful feedback.
2022.09, Back in Pittsburgh for a new semester! (+ CHI push)
2022.08, Submitted a paper to IMWUT. Thanks to my collaborators, especially for my advisor's last-minute help with Covid.
2022.07, "RGBDGaze" got accepted to ACM ICMI 2022. My first work at CMU!
2022.06, I am selected for Masason Foundation Scholarship .
2022.05, Moved to NYC for the exciting summer at Snap Research (computational imaging team)!
2022.05, Attended CHI'22 in NOLA. Everything was great except that I got Covid...(fortunately mild symptom).
2022.04, I'll attend TRAIT workshop at CHI'22 to present a case study of our conversational analysis project at ACES, Inc..
2022.03, "CalmResponses" got accepted to ACM IMX 2022. Congrats Kiyosu!
2022.01, I'll be serving as one of the Program Committee Members at ACM CHI LBW 2022.
2021.12, "BeParrot" got accepted to ACM IUI 2022.
2021.12, Visited Lifestyle Computing Lab. Honored and excited to introduce my projects.
2021.12, Visited UCLA. Great catching up with Anthony, Yang, and their students.
2021.11, Our patent of "REsCUE" is acquired by ACES, Inc.. link
2021.11, "VocabEncounter" got conditionally accepted to ACM CHI 2022.
2021.08, Moved to Pittsburgh, officially starting my Ph.D. journey.
2021.07, "Digital Speech Makeup" got accepted to ACM ICMI 2021.
2021.04, It's been a long journey and I'm thrilled to announce that I'll be joining CMU HCII this fall as a Ph.D. student!
2021.03, "Mindless Attractor" received Honorable Mention Award (top 5%) at ACM CHI 2021.
2021.03, Received Dean's Award and was selected for the representative student of the Graduate School of Information Science and Technology, The University of Tokyo.
2021.02, "Origami Reflector" received the Best Paper Award at the 1st CHIIoT Workshop!
2021.01, "Origami Reflector" got accepted to 1st CHIIoT Workshop.
2021.01, Submitted "Origami Reflector" to 1st CHIIoT Workshop. Thank you Yang Zhang for mentoring!
2020.12, "Reaction or Speculation" got conditionally accepted to ACM CSCW 2021.
2020.12, "Mindless Attractor" got conditionally accepted to ACM CHI 2021.
2020.11, I am selected for Funai Overseas Scholarship.
2020.08, "Hand with Sensing Sphere" got accepted to ACM SUI 2020.
2020.08, "Mimicker-in-the-Browser" got accepted to ACM ICMI 2020.
2020.03, "PenSight" received Best Paper Award (top 1%) at ACM CHI 2020.
2019.12, "INWARD" and "PenSight" got accepted to ACM CHI 2020.
2019.11, "Kininaruki" won Laval Virtual prize at International Virtual Reality Contest 2019.
2019.11, "BulkScreen" got accepted to ACM TEI Work-in-Progress 2020.
2019.08, "TransVoice" paper got accepted to ACM UIST Poster 2019.
2019.07, "DNN Real-time Voice Conversion" got accepted to The 10th ISCA Speech Synthesis Workshop.
2019.06, I received The 19th Best Student Presentation Award of The Acoustical Society of Japan, Sep. 2019. link
2019.04, "DQN-TAMER" got accepted to RT-DUNE workshop at IEEE ICRA 2019. link
2019.03, We exhibited "BulkScreen", a pin-array-based shape display, at SXSW 2019.
2019.01, I was featured as one Under30 person in NewsPicks. link
2018.12, "REsCUE" got accepted to ACM CHI 2019.
2018.10, "Your Pacifier" received the national grand prize in James Dyson Award 2018. link

Research

My research aims to develop high-impact intelligent systems with multimodal sensing/ML techniques and human-AI interaction perspectives. I research three themes by smartly digitizing user information (pose, gaze, context, physiological state, etc.) and designing trustful interventions.

how we use AIs to augment our everyday life [Interaction Techniques]
how we deploy AIs to real-world situations with the high need [Health]
how we design AIs that deal with highly human context [Social Signal Processing]

My inventions on social signal processing have been licensed to several companies in diverse fields such as executive coaching, sales enablement, online classroom, human assessment, and entertainment. I leverage such collaborations to study the actual deployment process of AI systems as an external researcher ("AI for human assessment [CHI’23 case study]" is an example).

Peer-Reviewed Conferences & Journals

LemurDx: Using Unconstrained Passive Sensing for an Objective Measurement of Hyperactivity in Children with no Parent Input

Riku Arakawa, Karan Ahuja, Kristie Mak, Gwendolyn Thompson, Sam Shaaban, Oliver Lindhiem, Mayank Goel

PACM IMWUT 2023 (Ubicomp) [To Appear]

CatAlyst: Domain-Extensible Intervention for Preventing Task Procrastination Using Large Generative Models

Riku Arakawa*, Hiromu Yakura*, Masataka Goto (*: equal contribution)

ACM CHI 2023

Paper, Slide, Video, Blog (Japanese), Blog (English)

IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds

Vimal Mollyn, Riku Arakawa, Mayank Goel, Chris Harrison, Karan Ahuja

ACM CHI 2023 (: Honorable Mention Award, top 5%)

Paper, Video, GitHub

uKnit: A Position-Aware Reconfigurable Machine-Knitted Wearable for Gestural Interaction and Passive Sensing using Electrical Impedance Tomography

Tianhong Catherine Yu, Riku Arakawa, James McCann, Mayank Goel

ACM CHI 2023

Paper, Video

PrISM-Tracker: A Framework for Multimodal Procedure Tracking Using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty

Riku Arakawa, Hiromu Yakura, Vimal Mollyn, Suzanne Nie, Emma Russell, Dustin P. Dimeo, Haarika A. Reddy, Alexander K. Maytin, Bryan T. Carroll, Jill Fain Lehman, Mayank Goel

PACM IMWUT 2022 (Ubicomp)

Paper, GitHub (coming soon)

RGBDGaze: Gaze Tracking on Smartphones with RGB and Depth Data

Riku Arakawa, Mayank Goel, Chris Harrison, Karan Ahuja

ACM ICMI 2022

Paper, Video, GitHub

VoLearn: A Cross-Modal Operable Motion-Learning System Combined with Virtual Avatar and Auditory Feedback

Chengshuo Xia, Xinrui Fang, Riku Arakawa, Yuta Sugiura

PACM IMWUT 2022 (Ubicomp)

Paper, Video

VocabEncounter: NMT-powered Vocabulary Learning by Presenting Computer-Generated Usages of Foreign Words into Users' Daily Lives

Riku Arakawa*, Hiromu Yakura*, Sosuke Kobayashi (*: equal contribution)

ACM CHI 2022

Paper, Slide, Video, Blog (Japanese), Blog (English)

BeParrot: Efficient Interface for Transcribing Unclear Speech via Respeaking

Riku Arakawa*, Hiromu Yakura*, Masataka Goto (*: equal contribution)

ACM IUI 2022

Paper, Slide, Video, Blog (Japanese), Blog (English)

Digital Speech Makeup: Voice Conversion Based Altered Auditory Feedback for Transforming Self-Representation

Riku Arakawa, Zendai Kashino, Shinnosuke Takamichi, Adrien Verhulst, Masahiko Inami

ACM ICMI 2021

Paper, Blog (Japanese)

Reaction or Speculation: Building Computational Support for Users in Catching-Up Series Based on an Emerging Media Consumption Phenomenon

Riku Arakawa*, Hiromu Yakura* (*: equal contribution)

PACM HCI 2021 (CSCW)

Paper, Slide, Blog (Japanese), Blog (English)

Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation

Riku Arakawa*, Hiromu Yakura* (*: equal contribution)

ACM CHI 2021 (: Honorable Mention Award, top 5%)

Paper, Video, GitHub Blog (Japanese) Blog (English)

Hand with Sensing Sphere: Body-Centered Spatial Interactions with a Hand-Worn Spherical Camera

Riku Arakawa, Azumi Maekawa, Zendai Kashino, Masahiko Inami

ACM SUI 2020

Paper, Video

INWARD: A Computer-Supported Tool for Video-Reflection Improves Efficiency and Effectiveness in Executive Coaching

Riku Arakawa*, Hiromu Yakura* (*: equal contribution)

ACM CHI 2020

Paper, Slide, Video, Blog (Japanese), Blog (English)

PenSight: Enhanced Interaction with a Pen-Top Camera

Fabrice Matulic, Riku Arakawa, Brian Vogel, Daniel Vogel

ACM CHI 2020 (: Best Paper Award, top 1%)

Paper, Video

REsCUE: A framework for REal-time feedback on behavioral CUEs using multimodal anomaly detection

Riku Arakawa*, Hiromu Yakura* (*: equal contribution)

ACM CHI 2019, Travel Grant by NEC C&C Foundation

Paper, Slide, Blog (Japanese), Blog (English)

Workshops, Posters, and Demos

AI for human assessment: What do professional assessors need?

Riku Arakawa*, Hiromu Yakura* (*: equal contribution)

ACM CHI 2023 Case Study (: Honorable Case Study Recognition, top 3)

Workshop on Trust and Reliance in AI-Human Teams (TRAIT’22) at CHI 2022

Paper Slide, Poster, Blog (Japanese)

Human-AI communication for human-human communication: Applying interpretable unsupervised anomaly detection to executive coaching

Riku Arakawa*, Hiromu Yakura* (*: equal contribution)

Communication in Human-AI Interaction Workshop (CHAI’22) at IJCAI 2022

Paper, Slide

Low-Cost Millimeter-Wave Interactive Sensing through Origami Reflectors

Riku Arakawa, Yang Zhang

1st CHIIoT Workshop (: Best Paper Award)

Paper, Video

Exploration of Reinforcement Learning for Event Camera using Car-like Robots

Riku Arakawa*, Shintaro Shiba* (*: equal contribution)

IEEE ICRA 2020 Unconventional Sensors in Robotics workshop

Paper, Video

BulkScreen: Saliency-Based Automatic Shape Representation of Digital Images with a Vertical Pin-Array Screen

Riku Arakawa*, Yudai Tanaka*, Hiromu Kawarasaki, Kiyosu Maeda (*: equal contribution)

ACM TEI 2020 Work-in-Progress

Paper, Video

TransVoice: Real-Time Voice Conversion for Augmenting Near-Field Speech Communication

Riku Arakawa, Shinnosuke Takamichi, Hiroshi Saruwatari

ACM UIST 2019 Poster

Paper, Poster

Implementation of DNN-based real-time voice conversion and its improvements by audio data augmentation and mask-shaped device

Riku Arakawa, Shinnosuke Takamichi, Hiroshi Saruwatari

The 10th ISCA Speech Synthesis Workshop

Paper, Video, Poster

DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedback

Riku Arakawa, Sosuke Kobayashi, Yuya Unno, Yuta Tsuboi, Shin-ichi Maeda

IEEE ICRA 2019 RT-DUNE workshop (Extended Abstract)

Paper, Poster

Honors and Awards

Fellowship, Scholarship and Grant-in-Aid

2023 Center for Machine Learning and Health, Translational Fellowship in Digital Health
2022 Snap Research Fellowship
2022 Masason Foundation Scholarship
2021 Funai Overseas Scholarship
2021 Kuma Creator Scholarship
2021 INNO-vation Program, Ministry of Internal Affairs and Communications, Japan
2021 Research Fellowship for Young Scientists (DC1), Japan Society for the Promotion of Science (until 2021.8)
2020 TOYOTA/Dwango AI Scholarship
2020 Fixstars Scholarship
2019 NEC C&C Grant for Attending International Conferences
2018 IPA Mitou Advanced Program, Ministry of Economy, Trade, and Industry, Japan

Academic Honors and Awards

2023.04 Honorable Mention Award, CHI2023 (top 5%) [IMUPoser]
2023.04 Honorable Case Study Recognition, CHI2023 (top 3) [AI for human assessment]
2021.03 Honorable Mention Award, CHI2021 (top 5%) [Mindless Attractor]
2021.03 Representative Student, Graduate School of Information Science and Technology at the University of Tokyo
2021.03 Dean's Award, Graduate School of Information Science and Technology at the University of Tokyo
2021.02 Best Paper Award, 1st CHIIoT workshop (top 1) [Origami Reflector]
2020.03 Best Paper Award, CHI2020 (top 1%) [PenSight]
2019.06 Best Student Presentation Award, The Acoustical Society of Japan [Real-Time Voice Conversion]
2015.03 Tokyo Governor Award (academic activities)
2015.03 Tokyo Private School Foundation Award (academic activities)

Programming & Science Competition Awards

2022.12 Finalist for INNO-vation Generation Award, Ministry of Internal Affairs and Communications, Japan [BeParrot]
2021.12 Finalist for INNO-vation Generation Award, Ministry of Internal Affairs and Communications, Japan [Mindless Attractor]
2019.11 International Virtual Reality Contest, Laval Virtual prize & Unity prize
2019.06 Spajam Hackathon Tokyo League, 1st prize
2018.10 James Dyson Award, Grand Prize in Japan
2018.02 RUNHACK, 1st prize
2017.11 JPHACKS, Innovator & three prizes
2017.10 Stanford Healthcare Hackathon, 3rd prize & design prize
2017.06 Jiro Hackathon, prize
2017.05 Primary Industries Hackathon, 1st prize
2017.03 London Deep Learning Hackathon, 1st prize & best pitch prize
2017.03 LINE Bot Awards, finalist and prize
2017.01 TwitCasting Hackathon, 1st prize
2016.11 HackerWars, prize
2016.07 AI chat bot hackathon, 1st prize
2015.08 Beijing IoT Competition, design prize
2014.09 Asian Science Camp, math prize

Profile

News

Research

LemurDx: Using Unconstrained Passive Sensing for an Objective Measurement of Hyperactivity in Children with no Parent Input

CatAlyst: Domain-Extensible Intervention for Preventing Task Procrastination Using Large Generative Models

IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds

uKnit: A Position-Aware Reconfigurable Machine-Knitted Wearable for Gestural Interaction and Passive Sensing using Electrical Impedance Tomography

PrISM-Tracker: A Framework for Multimodal Procedure Tracking Using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty

RGBDGaze: Gaze Tracking on Smartphones with RGB and Depth Data

VoLearn: A Cross-Modal Operable Motion-Learning System Combined with Virtual Avatar and Auditory Feedback

VocabEncounter: NMT-powered Vocabulary Learning by Presenting Computer-Generated Usages of Foreign Words into Users' Daily Lives

BeParrot: Efficient Interface for Transcribing Unclear Speech via Respeaking

Digital Speech Makeup: Voice Conversion Based Altered Auditory Feedback for Transforming Self-Representation

Reaction or Speculation: Building Computational Support for Users in Catching-Up Series Based on an Emerging Media Consumption Phenomenon

Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation

Hand with Sensing Sphere: Body-Centered Spatial Interactions with a Hand-Worn Spherical Camera

INWARD: A Computer-Supported Tool for Video-Reflection Improves Efficiency and Effectiveness in Executive Coaching

PenSight: Enhanced Interaction with a Pen-Top Camera

REsCUE: A framework for REal-time feedback on behavioral CUEs using multimodal anomaly detection

AI for human assessment: What do professional assessors need?

Human-AI communication for human-human communication: Applying interpretable unsupervised anomaly detection to executive coaching

Low-Cost Millimeter-Wave Interactive Sensing through Origami Reflectors

Exploration of Reinforcement Learning for Event Camera using Car-like Robots

BulkScreen: Saliency-Based Automatic Shape Representation of Digital Images with a Vertical Pin-Array Screen

TransVoice: Real-Time Voice Conversion for Augmenting Near-Field Speech Communication

Implementation of DNN-based real-time voice conversion and its improvements by audio data augmentation and mask-shaped device

DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedback

Honors and Awards

Fellowship, Scholarship and Grant-in-Aid

Academic Honors and Awards

Programming & Science Competition Awards

Talks (selected)

Media Coverage

Contact

Profile

News

Research

LemurDx: Using Unconstrained Passive Sensing for an Objective Measurement of Hyperactivity in Children with no Parent Input

CatAlyst: Domain-Extensible Intervention for Preventing Task Procrastination Using Large Generative Models

IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds

uKnit: A Position-Aware Reconfigurable Machine-Knitted Wearable for Gestural Interaction and Passive Sensing using Electrical Impedance Tomography

PrISM-Tracker: A Framework for Multimodal Procedure Tracking Using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty

RGBDGaze: Gaze Tracking on Smartphones with RGB and Depth Data

VoLearn: A Cross-Modal Operable Motion-Learning System Combined with Virtual Avatar and Auditory Feedback

VocabEncounter: NMT-powered Vocabulary Learning by Presenting Computer-Generated Usages of Foreign Words into Users' Daily Lives

BeParrot: Efficient Interface for Transcribing Unclear Speech via Respeaking

Digital Speech Makeup: Voice Conversion Based Altered Auditory Feedback for Transforming Self-Representation

Reaction or Speculation: Building Computational Support for Users in Catching-Up Series Based on an Emerging Media Consumption Phenomenon

Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation

Hand with Sensing Sphere: Body-Centered Spatial Interactions with a Hand-Worn Spherical Camera

INWARD: A Computer-Supported Tool for Video-Reflection Improves Efficiency and Effectiveness in Executive Coaching

PenSight: Enhanced Interaction with a Pen-Top Camera

REsCUE: A framework for REal-time feedback on behavioral CUEs using multimodal anomaly detection

AI for human assessment: What do professional assessors need?

Human-AI communication for human-human communication: Applying interpretable unsupervised anomaly detection to executive coaching

Low-Cost Millimeter-Wave Interactive Sensing through Origami Reflectors

Exploration of Reinforcement Learning for Event Camera using Car-like Robots

BulkScreen: Saliency-Based Automatic Shape Representation of Digital Images with a Vertical Pin-Array Screen

TransVoice: Real-Time Voice Conversion for Augmenting Near-Field Speech Communication

Implementation of DNN-based real-time voice conversion and its improvements by audio data augmentation and mask-shaped device

DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedback

Honors and Awards

Fellowship, Scholarship and Grant-in-Aid

Academic Honors and Awards

Programming & Science Competition Awards

Talks (selected)

Media Coverage

Contact

RGBDGaze: Gaze Tracking on Smartphones with RGB and Depth Data