You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello,I am doing a research about Active Learning under Spark,When I use idea to process my code,i have the same issue. I am waiting for resolve this problem two days.Can you tell me about the answer?
#628
Open
ViViBao opened this issue
May 29, 2024
· 0 comments
import org.apache.spark.sql.{SparkSession, DataFrame}
import org.apache.spark.ml.classification.{DecisionTreeClassifier, DecisionTreeClassificationModel}
import org.apache.spark.ml.feature.VectorAssembler
import org.apache.spark.ml.linalg.Vector
import org.apache.spark.sql.functions._
object ActiveLearningExample {
def main(args: Array[String]): Unit = {
val spark = SparkSession.builder()
.appName("ActiveLearningExample")
.master("local[*]")
.config("spark.driver.host", "localhost")
.getOrCreate()
}
}
object ActiveLearningStrategy {
import org.apache.spark.ml.linalg.Vector // 确保导入正确的Vector类型
def calculateEntropy(probabilities: Vector): Double = {
probabilities.toArray.map(p => if (p == 0) 0 else -p * Math.log(p)).sum
}
def selectSamplesForLabeling(data: DataFrame, model: DecisionTreeClassificationModel, k: Int): DataFrame = {
val predictions = model.transform(data)
}
}
The text was updated successfully, but these errors were encountered: