Tuesday, 11 July 2023

Educational Data Mining

 ...

Referensi

  1. Data mining applications in university information management system development, https://www.degruyter.com/document/doi/10.1515/jisys-2022-0006/html
  2. Educational data mining: prediction of students' academic performance using machine learning algorithms, https://slejournal.springeropen.com/articles/10.1186/s40561-022-00192-z#Tab2
  3. Data Mining on the Prediction of Student’s Performance at the High School National Examination, https://www.scitepress.org/Papers/2021/104080/104080.pdf
  4. National standardized tests database implemented as a research methodology in mathematics education. The case of algebraic powers, https://hal.science/hal-02430515/document
  5. Student Performance in R, https://github.com/nicolettejohnson/student-performance-r
  6. Use Data Warehouse and Data Mining to Predict Student Academic Performace in Schools,, https://www.slideshare.net/Ranjithgowda93/data-mining-to-predict-academ

Tuesday, 21 March 2023

Applied Multivariate Analysis with R in dairy farming



Applied multivariate analysis with R can be useful in dairy farming for a variety of purposes, such as:

1. Cluster analysis 


Cluster analysis can be used to group cows based on their milk production, milk composition, and other characteristics. This can help farmers identify subgroups of cows that require specific management practices, such as different feed or medication regimes.

Here's an example of how to perform cluster analysis using multivariate analysis with R in dairy farming:

# Load the necessary packages
library("cluster")
library("factoextra")

# Load the dataset (replace "data.csv" with the name of your file)
data <- read.csv("data.csv")

# Select the variables you want to use for clustering (replace "var1", "var2", etc. with the names of your variables)
vars <- data[,c("var1", "var2", "var3", "var4")]

# Perform hierarchical clustering
hc <- hclust(dist(vars))

# Determine the optimal number of clusters using the elbow method
fviz_nbclust(vars, hcut, method = "wss") # "wss" stands for "within sum of squares"

# Based on the elbow plot, let's say we choose 4 clusters
k <- 4

# Perform k-means clustering
km <- kmeans(vars, k)

# Visualize the clusters
fviz_cluster(km, data = vars, stand = FALSE, geom = "point")

In this example, we first load the necessary packages (cluster and factoextra) and then load the dataset. We then select the variables we want to use for clustering and perform hierarchical clustering using the hclust() function. We then use the fviz_nbclust() function from the factoextra package to determine the optimal number of clusters using the elbow method. Based on the elbow plot, we choose 4 clusters and then perform k-means clustering using the kmeans() function. Finally, we visualize the clusters using the fviz_cluster() function from the factoextra package.

Note that you will need to modify this code to fit your specific dataset and research question.

2. Principal Component Analysis (PCA)


PCA can be used to identify patterns and relationships among variables that contribute to milk production. This can help farmers identify key factors that impact milk production and develop strategies to optimize these factors.

Here's an example of how to perform Principal Component Analysis (PCA) using multivariate analysis with R in dairy farming:

# Load the necessary packages
library("FactoMineR")
library("factoextra")

# Load the dataset (replace "data.csv" with the name of your file)
data <- read.csv("data.csv")

# Select the variables you want to use for PCA (replace "var1", "var2", etc. with the names of your variables)
vars <- data[,c("var1", "var2", "var3", "var4")]

# Perform PCA
pca <- PCA(vars, graph = FALSE)

# Visualize the results
fviz_pca_var(pca) # plot of variables
fviz_pca_biplot(pca) # biplot of variables and observations


In this example, we first load the necessary packages (FactoMineR and factoextra) and then load the dataset. We then select the variables we want to use for PCA and perform PCA using the PCA() function from the FactoMineR package. We set graph = FALSE to prevent the function from automatically plotting the results. Finally, we visualize the results using the fviz_pca_var() and fviz_pca_biplot() functions from the factoextra package.

Note that you will need to modify this code to fit your specific dataset and research question. Additionally, you may want to explore other options for visualizing the results of PCA, such as scree plots, heatmaps, or 3D scatterplots.

3. Discriminant Analysis


Discriminant analysis can be used to classify cows based on their milk production, milk composition, or other characteristics. This can help farmers identify which cows are the most productive and which may need additional attention.

Here's an example of how to perform Discriminant Analysis using multivariate analysis with R in dairy farming:

# Load the necessary packages
library("MASS")
library("caret")

# Load the dataset (replace "data.csv" with the name of your file)
data <- read.csv("data.csv")

# Split the dataset into training and testing sets (replace "0.8" with the proportion of data you want to use for training)
index <- createDataPartition(data$Class, p = 0.8, list = FALSE)
train <- data[index,]
test <- data[-index,]

# Select the variables you want to use for discriminant analysis (replace "var1", "var2", etc. with the names of your variables)
vars <- train[,c("var1", "var2", "var3", "var4")]

# Perform linear discriminant analysis
lda <- lda(Class ~ ., data = train[,c("Class", vars)])

# Predict the classes of the testing set
predictions <- predict(lda, test[,c("var1", "var2", "var3", "var4")])

# Evaluate the accuracy of the predictions
confusionMatrix(predictions$class, test$Class)

In this example, we first load the necessary packages (MASS and caret) and then load the dataset. We then split the dataset into training and testing sets using the createDataPartition() function from the caret package. We select the variables we want to use for discriminant analysis and perform linear discriminant analysis using the lda() function from the MASS package. We then predict the classes of the testing set using the predict() function and evaluate the accuracy of the predictions using the confusionMatrix() function from the caret package.

Note that you will need to modify this code to fit your specific dataset and research question. Additionally, you may want to explore other options for performing discriminant analysis, such as quadratic discriminant analysis or regularized discriminant analysis.


4. Regression Analysis


Regression analysis can be used to model the relationship between milk production and various predictors, such as age, breed, diet, and management practices. This can help farmers identify the factors that contribute to milk production and develop strategies to optimize these factors.

Here's an example of how to perform Regression Analysis using multivariate analysis with R in dairy farming:

# Load the necessary packages
library("car")
library("tidyverse")

# Load the dataset (replace "data.csv" with the name of your file)
data <- read.csv("data.csv")

# Select the variables you want to use for regression analysis (replace "var1", "var2", etc. with the names of your variables)
vars <- data[,c("var1", "var2", "var3", "var4")]

# Fit a multiple linear regression model
model <- lm(outcome ~ var1 + var2 + var3 + var4, data = data)

# Check the assumptions of the model
plot(model) # plot of residuals vs. fitted values
qqPlot(model) # normal probability plot of residuals

# Evaluate the performance of the model
summary(model) # summary of model coefficients and significance
confint(model) # confidence intervals of model coefficients
anova(model) # analysis of variance table

# Make predictions using the model
new_data <- data.frame(var1 = c(1, 2, 3), var2 = c(4, 5, 6), var3 = c(7, 8, 9), var4 = c(10, 11, 12))
predictions <- predict(model, newdata = new_data)


In this example, we first load the necessary packages (car and tidyverse) and then load the dataset. We then select the variables we want to use for regression analysis and fit a multiple linear regression model using the lm() function from the stats package. We check the assumptions of the model using the plot() and qqPlot() functions from the car package. We evaluate the performance of the model using the summary(), confint(), and anova() functions. Finally, we make predictions using the model by creating a new dataset with the predictor variables and using the predict() function.

Note that you will need to modify this code to fit your specific dataset and research question. Additionally, you may want to explore other options for performing regression analysis, such as non-linear regression, mixed-effects models, or generalized linear models.


5. Time Series Analysis


Time series analysis can be used to forecast future milk production based on historical data. This can help farmers plan for future milk production and make informed decisions about pricing, marketing, and other business decisions.

Here's an example of how to perform Time Series Analysis using multivariate analysis with R in dairy farming:

# Load the necessary packages
library("zoo")
library("ggplot2")

# Load the dataset (replace "data.csv" with the name of your file)
data <- read.csv("data.csv")

# Convert the data to a time series object
ts_data <- zoo(data[,c("var1", "var2", "var3", "var4")], order.by = data$Date)

# Plot the time series
autoplot(ts_data, facets = TRUE) + theme_minimal()

# Decompose the time series
decomp <- decompose(ts_data)
autoplot(decomp)

# Fit a time series model
model <- auto.arima(ts_data$var1)

# Make predictions using the model
predictions <- forecast(model, h = 10)

# Plot the predictions
autoplot(predictions) + theme_minimal()


In this example, we first load the necessary packages (zoo and ggplot2) and then load the dataset. We convert the data to a time series object using the zoo() function from the zoo package and plot the time series using the autoplot() function from the ggplot2 package. We then decompose the time series using the decompose() function and plot the components using autoplot(). We fit a time series model using the auto.arima() function from the forecast package and make predictions using the forecast() function. Finally, we plot the predictions using autoplot().

Note that you will need to modify this code to fit your specific dataset and research question. Additionally, you may want to explore other options for performing time series analysis, such as seasonal ARIMA models, exponential smoothing models, or dynamic regression models.

In summary, multivariate analysis with R can be a powerful tool for dairy farmers to optimize their production practices and improve their profitability.


References

1. ChatGPT. (2023, March 21). Applied Multivariate Analysis with R in dairy farming  [Online forum post]. Retrieved from https://www.gpt.com

Tuesday, 14 June 2022

Factor Analysis

SPSS

Analyze --> Dimension Reduction --> Factor 


Extraction --> Method (Maximum likelihood)

  Referensi

  1. SPSS Factor Analysis – Beginners Tutorial, https://www.spss-tutorials.com/spss-factor-analysis-tutorial/

Wednesday, 7 October 2020

Assessing ranks

Pertanyaan  

Misalkan, jika kita memiliki nilai Fisika dari seluruh sekolah yang berada di sebuah Desa, bagaimana cara merangking sekolah berdasarkan capaian kemampuan Fisikanya, jika populasi siswa setiap sekolah sangat berbeda, misal ada sekolah yang mengambil ujian Fisika hanya 5 orang dan ada sekolah lain yang sampai 100 orang. Karena kalau merangking dari nilai rata-rata saja, tidak begitu valid, terutama jika populasinya kecil. 



Referensi 

  1. An Introduction to statistics : Assessing ranks, http://www.floppybunny.org/robin/web/virtualclassroom/stats/basics/part8.pdf
  2. Score Transformations, http://faculty.cbu.ca/~erudiuk/IntroBook/sbk14m.htm
  3. Percentile rank scores are congruous indicators of relative performance, or aren’t they?, https://arxiv.org/pdf/1108.1860.pdf
  4. Comparing Means in R, http://www.sthda.com/english/wiki/comparing-means-in-r
  5. Better Ranking using Bayesian Average, https://arpitbhayani.me/blogs/bayesian-average
  6. Analisis Kruskall-Wallis dengan SPSS,  https://www.semestapsikometrika.com/2018/11/analisis-kruskall-wallis-dengan-spss.html
  7. One-Way Analysis of Variance:Comparing Several Means, https://www.westga.edu/academics/research/vrc/assets/docs/OneWayANOVA_LectureNotes.pdf
  8. Compare the means of two or more variables or groups in the data, https://radiant-rstats.github.io/docs/basics/compare_means.html 
  9.  Using the Bayesian Average in Custom Ranking, https://www.algolia.com/doc/guides/managing-results/must-do/custom-ranking/how-to/bayesian-average/
  10. Use Bayesian Averages to Improve Rating Sorting in your Elasticsearch Index, https://jolicode.com/blog/use-bayesian-averages-to-improve-rating-sorting-in-your-elasticsearch-index
  11. Bayesian Average Ratings, https://www.evanmiller.org/bayesian-average-ratings.html
  12. POSTGRESQL: WEIGHTED AVERAGE INSTEAD OF AVERAGE?-POSTGRESQL, https://www.appsloveworld.com/postgresql/100/95/postgresql-weighted-average-instead-of-average

Saturday, 26 October 2019

WrightMap: IRT Item-Person Map

Secara default, pelabelan item dalam item person map menggunakan WrightMap adalah berdasarkan urutan. Jika ada soal yang dihapus, maka, akan membingungkan membaca item-person map nya.

David sudah menambah fitur untuk menggunakan custom label ini, seperti dijelaskan disini [2], tapi sepertinya update ini, baru bisa digunakan jika input WrightMap berupa CQModel

Referensi


  1. WrightMap: IRT Item-Person Map with 'ConQuest' Integration, https://cran.r-project.org/web/packages/WrightMap/index.html
  2. Using item names as item.labels #6, https://github.com/david-ti/wrightmap/issues/6

Arti Nilai Parameter b yang Terlampau Besar pada Teori IRT

...

Referensi


  1. Meaning of large values of parameter b (|b| >>) 4 in IRT theory, https://stats.stackexchange.com/questions/169247/meaning-of-large-values-of-parameter-b-b-4-in-irt-theory