2022年度ベンチャー体験工房3
Undergraduate School and Graduate School
学部・大学院
- ベンチャー体験工房3
- ノイズが多く不規則な福島の交通渋滞ビッグデータを保存するためのテラバイト規模のデータウェアハウスの設計
- JApan Road Transportation and Information Center (JARTIC) has set up a nationwide sensor network to monitor traffic congestion. This network generates traffic congestion data at 5-minute intervals (in some cases at 1-minute intervals). The generated data represents noisy and irregular spatiotemporal big data. We have purchased traffic congestion data for the entire Fukushima prefecture for the time duration April-2019 to March-2020. Analyzing the data of this time is important because it covers the key events constituting of Typhoon Hagibis and Covid emergence. In this course, the students will focus on developing Big Data Warehouse for storing and processing the huge real-world traffic congestion data.
- シラバス詳細
2022年10月
10月11日
授業内容： Fukushima system understanding and collecting the data in Excel sheets.
We got to know how the data is collected in real world. And how sensors helps us in collecting the data. The collection of data done in real world is really interesting.
10月17日
授業内容：Understanding the data in Excel sheets and designing ER schema accordingly.
Learnt how to use and control the data using technology. And also learnt different skills like Excel, Database design (ER schema).
10月24日
授業内容：Creating Postgresql tables for ER schema designed and inserting the data into tables.
Excited to express how large amounts of data can be fitted into tables using postgresql. It really wonders us.
2022年11月
11月2日
授業内容：Any pending works of previous month and creation of csv file(dataframe) -specified format: traffic congestion length in a given location at a particular time.
Had a good experience on working with traffic congestion data. Able to understand how data is created and analysis is done.
11月7日
授業内容：Checking whether there are any missing pixels in the created data frame and performing imputation on the data if missing pixels are present. Evaluating error for different imputation methods.
Nice Experience on learning imputation topic with many techniques.
2022年12月
12月12日
授業内容： Complete Linear Regression Techniques. Start learning Time Series Forecasting models. Apply them to the given traffic congestion Dataset
Learnt Forecasting methods.
12月19日
授業内容： Learn Walk Forward Validation Technique. Apply walk forward validation technique and evaluate results.
Applied Time Series Forecasting to the real-world data. Nice Experience.
2023年1月
1月16日
授業内容：Wrapping up timeseries forecasting and Studying about LSTM.
Learnt LSTM.
1月23日
授業内容： Applying LSTM to predict the congestion
Applied LSTM.
1月30日
授業内容：Tried to reduce the error rate using different parameters and normalizing the data.
Able to predict congestion values with less error rate.