Assignment 2 - Text data

As a group with Manning and Pan, I was in charge with data finding and basic processing.

Idea building

When we talk about text data, we talk about meaning, communication, emotion, and expression. Since all of us are fans of the TV show <Friends>, we thought about using the scripts of this show as our data source. We were thinking about different kinds of data visualization of this script, such as places they went to in NY, relationship or intimacy between these six characters according to time, and personal characteristic analyzing from each one's words choosing. But finally we decided to try something different from visualization, vocalization.

Then we had this conversation about creature communication with each other without words, like bird. So what is going to be like when we have no text and talk like a bird in a super abstract way, can we express ourselves very well? Can we sense the emotion from each other?

Finally we settled down our idea about "script vocalization in a abstract way".

For this assignment I really want to treat it as a chance to learn node.js with this sentiment library . I failed before our presentation, so I did data processing in Excel, again, for my team. After that I spent more time on it and figure it out as below.

Procedure

1. Data collection

I kept this 10 seasons <Friends> script for a long time. It's a large quantity of data, so we choose on episode, episode 10 in season 7, as our data source.

2. Basic data analyzing

1) clean up raw data

data1.jpg
data2.jpg

2) analyse sentences' sentiment in Node.js and export result to team member so that they can play music based on character name and sentiment scores in p5.js.

var sentiment = require(‘sentiment’);
var fs = require(‘fs’);
var path = require(‘path’);

var filePathRead = path.join(__dirname, ‘friends.txt’);
var filePathWrite = path.join(__dirname, ‘friends-out.txt’);


var readStream = fs.createReadStream(filePathRead, ‘utf8’);
var writeStream = fs.createWriteStream(filePathWrite,’utf8’);


//read file and split it to myData[][]
readStream.on(‘data’, function (chunk) {
var myData = new Array();
var data = chunk.toString().split(‘\r’);
readStream.pause();
for (var i = 0; i < data.length; i++) {
//console.log(“data[“+i+”]:” + data[i]+”\r\n”);
myData[i] = new Array();
myData[i] = data[i].split(“:”);
//console.log(myData[i][0]); //all the [][0] is the name of scentence speaker
// for(var j = 0;j// console.log(“myData[“+i+”][“+j+”]:”+myData[i][j]+”\r\n”);
// }
}
getSentiment(myData);
readStream.resume();
});

readStream.on(‘end’, function(){
//console.log(“length of myData:”+myData.length);
writeStream.close();
})

function getSentiment(data){
for(var i = 0; i for(var j = 1;j data[i][j]=sentiment(data[i][j]).score;
}
writeStream.write(data[i]+”\r\n”);
}
}
data3.jpeg

summery

In this project, I went through the whole Node.js basic tutorial which is super helpful for my future skill set.