bdewilde.github.io - ← previous

Burton DeWilde

Bdewilde.github.io Spined HTML

Burton DeWilde Burton DeWildeWell-nighMe Archive CV Intro to Natural Language Processing (1) 2012-12-16 Harmony Institute information extraction natural language processing zombies First, the big news: I got a job! I’m now a data scientist at a non-profit organization here in Manhattan tabbed Harmony Institute, where we study the science of influence through entertainment. Basically, simple metrics like box office sales and television viewers don’t ratherish quantify a mucosa or show’s social impact; we use theory-driven methodology to increasingly fully assess this impact in individuals and wideness networks. In the specimen of, say, a social justice documentary that was made specifically to have such an impact, we are worldly-wise to quantify the film’s level of success.Increasinglyon this over time, I’m sure. ReadIncreasingly» Web Scraping and HTML Parsing (2) 2012-11-26 HTML parsing Metacritic natural language processing PyCon web scraping Wikipedia As I wrote last time, the Internet is stuffed-up of data, but much of it is “messy” and unstructured and spread throughout an HTML tree — in other words, not ready for analysis. Fortunately, web scraping and HTML parsing indulge for the streamlined extraction of online data and its conversion into a increasingly analysis-friendly form; unfortunately, it can be an villainous lot of work. In fact, data scientists often spend increasingly of their time getting and cleaning data than analyzing it! ReadIncreasingly» Web Scraping and HTML Parsing (1) 2012-11-14 BeautifulSoup HTML parsing HTTP Python Scrapy web scraping Hi! I haven’t posted in a while: I got displaced by Sandy, distracted by job applications, and overrun by zombies. It happens. But when to business… ReadIncreasingly»Nomenclatureof Hand-written Digits (4) 2012-10-29 nomenclature cross-validation hand-written digits Kaggle kNN R In my previous posts (Part 1 | Part 2 | Part 3), I described the k-nearest neighbors algorithm, unromantic a benchmark model to the nomenclature of hand-written digits, then chose an optimal value for k as the one that minimized the model’s prediction error on a defended validation data set. I moreover excluded well-nigh 2/3 of the features (image pixels) from the model considering they had near-zero variance, thereby improving both performance and runtime. Now, I’d like to add one last multiplicity to the kNN model: weighting. ReadIncreasingly»Nomenclatureof Hand-written Digits (3) 2012-10-26 nomenclature hand-written digits Kaggle kNN machine learning optimization R Now for the fun part! In Part 1, I described the machine learning task of nomenclature and some well-known examples, such as predicting the values of hand-written digits from scanned images. In Part 2, I outlined a unstipulated wringer strategy and visualized the training set of hand-written digits, gleaning at least one useful insight from that. Now, in Part 3, I pick a learning algorithm, train and optimize a model, and make predictions well-nigh new data! ReadIncreasingly» A Shout-out for Liberal Arts 2012-10-22 domain expertise KalamazooHigherliberal arts education machine learning This past weekend I was at KalamazooHigherfor my five-year higher reunion. I mentally prepared myself to finger really old but came out of it feeling young and refreshed instead. Funny how that works out. ReadIncreasingly»Nomenclatureof Hand-written Digits (2) 2012-10-17 wringer strategy nomenclature data viz hand-written digits machine learning optimization InNomenclatureof Hand-written Digits (1), I qualitatively described the machine learning task of nomenclature and sketched out two archetype examples, then went into increasingly detail well-nigh flipside well-known example: the nomenclature of hand-written digits. The rencontre here is to program a classifier that correctly predicts the value represented in a scanned image of a hand-written digit. ReadIncreasingly»Nomenclatureof Hand-written Digits (1) 2012-10-14 nomenclature hand-written digits iris dataset Kaggle supervised learning In the last few posts, I’ve attempted to lay a vital foundation explaining what data science is often about; in the next few posts, I’d like to delve deeper into a specific example. ReadIncreasingly» VersionTenancyIs Important! 2012-10-10 replacement branching collaboration Git Kaggle rewinding SVN version tenancy I recently came wideness a post on Kaggle’s no self-ruling hunch blog, “Engineering Practices in Data Science”, in which Chris Clark describes a set of weightier practices for those who work in the medium of lawmaking — specifically, those practices worldwide among software engineers but not among data scientists. I was much chagrined by the first wag of his finger: many data scientists don’t use version tenancy (a logical way to manage multiple versions of the same information, e.g. source code), preferring instead to save files with elaborate names and/or when them up dropbox. Ugh, I realized, he’s talking well-nigh me. ReadIncreasingly» Data asGravityfor PositiveTranspiration2012-10-06 big data DataKind positive transpiration practice I’ve been reading a lot well-nigh how big data is or is going to revolutionize the world we live in. Yes, some of this is hype; yes, increasingly data ways increasingly potential for statistical shenanigans and bad analysis; and yes, some changes may not necessarily be for the largest (think: violation of privacy and other unethical abuses of personal data). But it’s important to recognize that data can be a powerful gravity for positive transpiration in the world. ReadIncreasingly» ← previous ↑ next → Burton DeWilde data scientist / physicist / filmmaker © 2014 Burton DeWilde. All rights reserved.