AYAOSHIMA

Works: Machine Learning and Neural Modeling

Tensor Decomposition and Information-Theoretic Modeling of High-Dimensional Neural Time Series

  • Processed PVN neural recordings by spike sorting, event-based binning, z-scoring, and construction of neuron-time-trial tensors capturing trial-aligned spike dynamics.
  • Applied Tensor Component Analysis (TCA) with CP-ALS and Non-Negative CP decompositions to extract low-dimensional neuron, time, and trial factors, revealing engagement-modulated latent structures.
  • Estimated effective connectivity using Transfer Entropy to model engagement-dependent changes in directional information flow between neurons.
  • Found that active engagement significantly enhanced temporal coordination, neural synchrony, and trial-to-trial representational stability in PVN circuits.
  • Built the full modeling pipeline in MATLAB and Python, integrating tensor optimization, statistical validation, smoothing, and high-dimensional time series processing.

Tensor DecompositionTCATime Series ModelingDimensionality ReductionTransfer EntropySpike SortingNeural Data AnalysisMATLABPythonComputational ModelingStatistical Modeling
Tensor Decomposition and Information-Theoretic Modeling of High-Dimensional Neural Time Series Image 1Tensor Decomposition and Information-Theoretic Modeling of High-Dimensional Neural Time Series Image 2

Low-Latency Voice-to-NLP Recommender: Real-Time ASR, NLP/NLU, and Neural Search

  • Built a production-grade voice AI system that transcribes natural speech and delivers real-time personalized workout recommendations with <0.2s end-to-end latency.
  • Fine-tuned DistilBERT achieving 97.8% intent classification accuracy (top-1) and fine-tuned a RoBERTa-based spaCy Transformer NER model reaching 98.5% macro-averaged F1-score on noisy ASR-transcribed input.
  • Optimized OpenSearch retrieval with custom boosting strategies to maximize relevance and responsiveness for natural voice queries.
  • Built a scalable, asynchronous FastAPI backend and a lightweight React frontend optimized for instant voice query interaction.
  • Systematically tracked model training, evaluation metrics, and deployment iterations with MLflow, ensuring reproducibility and experiment management.

WhisperASRDistilBERTIntent ClassificationNERspaCyOpenSearchFastAPIReactPython
Low-Latency Voice-to-NLP Recommender: Real-Time ASR, NLP/NLU, and Neural Search YouTube thumbnail

Neural-Symbolic VQA: Multi-Modal AI with CNN, NLP, and Symbolic Reasoning

  • Developed a Neural-Symbolic VQA system, integrating computer vision (CNN), NLP, and Symbolic Reasoning, trained and evaluated on the Sort-of-CLEVR dataset, which includes 10k images and 200k questions.
  • Achieved accuracy of 88% in relational and 99% in non-relational questions.

Multi-ModalNLPComputer VisionCNNPyTorchSymbolic ReasoningPython
Neural-Symbolic VQA: Multi-Modal AI with CNN, NLP, and Symbolic Reasoning Image 1

Object Tracking Under Occlusions: CNN Detection with Kalman Prediction

Leveraged a pre-trained CNN (ResNet), combined with the MS COCO dataset, resulting in accurate tracking of obscured ball trajectories, integrating object detection, classification, and motion prediction using Kalman filter.

Mathematical ModelingKalman FilterObject TrackingCNNResNetPython
Object Tracking Under Occlusions: CNN Detection with Kalman Prediction YouTube thumbnail