Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Edge cases #3

Open
wants to merge 37 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
f0438a2
updates to run with spark 1.4.1
Nov 19, 2015
d5affba
path fixes
Nov 19, 2015
e3f15ba
missing dep
Nov 19, 2015
ce086da
resolver
Nov 19, 2015
4a604a5
updates to run with spark 1.4.1
Nov 19, 2015
922f9c4
proceed to next failure point
Nov 19, 2015
290cf15
missing dep
Nov 19, 2015
d21538f
type the accumulator
Nov 19, 2015
aa78255
looks like we can remove the workaround for Broadcast / HttpBroadcast
Nov 19, 2015
1c2dbc4
import
Nov 19, 2015
118e095
fix serializer path, give some default inputs
Nov 22, 2015
cbf7f1f
fix serializer path, give some default inputs
Nov 22, 2015
a5b4a64
capitalization
Nov 22, 2015
c5c4d82
this should really by dynamic based on scala version
Nov 22, 2015
7d34da9
looks like someone was building on a case insensitive fs
Nov 22, 2015
fab5e8f
insisitent variable naming... did this ever work?!?
Nov 22, 2015
eebf8d8
increase max fragment length
Nov 26, 2015
e13d8d5
errors. presumably unseen edge conditions
Nov 26, 2015
f9c18f0
errors. presumably unseen edge conditions
Nov 26, 2015
d742edd
errors. presumably unseen edge conditions
Nov 26, 2015
01567b3
clean comments
Nov 29, 2015
ecbd718
comment cleanup
Nov 29, 2015
3a5fe4b
comment cleanup
allenday Nov 30, 2015
a4fd208
errors. presumably unseen edge conditions
Nov 26, 2015
8b804ad
errors. presumably unseen edge conditions
Nov 26, 2015
53fe113
errors. presumably unseen edge conditions
Nov 26, 2015
d967970
increase max fragment length
Nov 26, 2015
318c7dc
comment cleanup
allenday Nov 30, 2015
46b287a
comment cleanup
allenday Nov 30, 2015
21b030d
allow very long fragments. should we filter these in ExpressD.scala …
allenday Nov 30, 2015
3b11ffc
no set seed
allenday Dec 6, 2015
5a1000a
no set seed
allenday Dec 6, 2015
1cefe4f
restore var rand, but without seed
allenday Dec 8, 2015
c98f14f
restore var rand, but without seed
allenday Dec 8, 2015
eb0c46b
sync with upstream changes
Jan 13, 2016
bac6379
still working on edge cases
Jan 13, 2016
669353d
resolve conflict
Jan 13, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 22 additions & 10 deletions src/main/scala/expressd/ExpressD.scala
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,9 @@ object ExpressD {
val LOG_1 = math.log(1)
val INF = 1.0/0.0

var rand = new Random(42)
//set seed for identical results between runs, e.g. Random(42)
//uncomment this for identical results between runs
var rand = new Random()

// Starts, Lengths, mismatchIndices, mismatchNucs
type ReadAlignments = (
Expand Down Expand Up @@ -881,6 +883,7 @@ object ExpressD {
val left = fragment._3
val right = fragment._4


for (i <- 0 until numAlignments) {
val pairTargetId = fragment._1(i)
val leftFirst = fragment._2(i)
Expand All @@ -902,7 +905,8 @@ object ExpressD {
val bcEffLengthsValue = bcEffLengths.value(pairTargetId)

likelihoods.set(i,
bcTaus.value(pairTargetId) + bcFld.value(pairLength) - bcEffLengths.value(pairTargetId))
bcTausValue + bcFldValue - bcEffLengthsValue
)

val targSeq = bcTargSeqs.value(pairTargetId)
val targLen = bcTargLens.value(pairTargetId)
Expand All @@ -927,14 +931,16 @@ object ExpressD {
}

if (shouldUpdateAllParams) {
errorIndices1.add(if (errorIndex1 == null) { errorIndices1.get(0) } else { errorIndex1 })
errorIndices2.add(if (errorIndex2 == null) { errorIndices2.get(0) } else { errorIndex2 })
errorIndices1.add(if (errorIndex1 == null) { if (errorIndices1.size > 0) { errorIndices1.get(0) } else { errorIndex1 } } else { errorIndex1 })
errorIndices2.add(if (errorIndex2 == null) { if (errorIndices2.size > 0) { errorIndices2.get(0) } else { errorIndex2 } } else { errorIndex2 })
}

// Sum current likelihood with values in errors likelihood table at MAX_READ_LENGTH indices.
var errLikelihood1 = LOG_1
if (i > 0 && errorIndex1 == null) {
errLikelihood1 = refErrLikelihood1
} else if ( errorIndex1 == null ) {
//TODO: what to do here? was throwing NPE here -aday
} else {
for (j <- 0 until errorIndex1.size) {
val splitCode = decodeMarkovChainIndex(errorIndex1(j))
Expand All @@ -947,6 +953,8 @@ object ExpressD {
var errLikelihood2 = LOG_1
if (i > 0 && errorIndex2 == null) {
errLikelihood2 = refErrLikelihood2
} else if ( errorIndex2 == null ) {
//TODO: what to do here? was throwing NPE here -aday
} else {
for (j <- 0 until errorIndex2.size) {
val splitCode = decodeMarkovChainIndex(errorIndex2(j))
Expand Down Expand Up @@ -1039,13 +1047,17 @@ object ExpressD {
val errorIndex2 = errorIndices2.get(i)

newFld.localValue(pairLength) += p
for (j <- 0 until errorIndex1.size) {
val splitCode = decodeMarkovChainIndex(errorIndex1(j))
newErrors1.localValue(j)(splitCode._1)(splitCode._2) += p
if ( errorIndex1 != null ) { //TODO is this right? avert NPE -aday
for (j <- 0 until errorIndex1.size) {
val splitCode = decodeMarkovChainIndex(errorIndex1(j))
newErrors1.localValue(j)(splitCode._1)(splitCode._2) += p
}
}
for (j <- 0 until errorIndex2.size) {
val splitCode = decodeMarkovChainIndex(errorIndex2(j))
newErrors2.localValue(j)(splitCode._1)(splitCode._2) += p
if ( errorIndex2 != null ) { //TODO is this right? avert NPE -aday
for (j <- 0 until errorIndex2.size) {
val splitCode = decodeMarkovChainIndex(errorIndex2(j))
newErrors2.localValue(j)(splitCode._1)(splitCode._2) += p
}
}

if (shouldUseBias) {
Expand Down
2 changes: 1 addition & 1 deletion src/main/scala/expressd/ExpressEnv.scala
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ object ExpressEnv {

val BIAS_WINDOW = System.getProperty("express.biasWindow", "8").toInt
val BIAS_MATRIX_SIZE = BIAS_WINDOW * 2 + 1
val MAX_FRAG_LENGTH = System.getProperty("express.maxFragLength", "800").toInt
val MAX_FRAG_LENGTH = System.getProperty("express.maxFragLength", "1000000").toInt
val MAX_READ_LENGTH = System.getProperty("express.maxReadLength", "200").toInt

val BURN_IN_ITERATIONS = System.getProperty("express.burnInIterations", "20").toInt
Expand Down