Very large database insert fails - MySQL, JPA 2, WildFly 8 -

Very large database insert fails - MySQL, JPA 2, WildFly 8 -

mysql community server v5.6.19 jpa 2 eclipselink wildfly 8 java ee 7 jdk 1.8 container managed persistence

i'm trying load big amount of reference info xml file , store in mysql database.

in total there approximately 60,000 records need written db.

all first 7,500 records or so, following:-

00:06:37,157 warn [com.arjuna.ats.arjuna] (transaction reaper) arjuna012117: transactionreaper::check timeout tx 0:ffff0a00000b:2d6361a7:5435c20e:22 in state run 00:06:37,161 warn [com.arjuna.ats.arjuna] (transaction reaper worker 0) arjuna012095: abort of action id 0:ffff0a00000b:2d6361a7:5435c20e:22 invoked while multiple threads active within it. 00:06:37,161 warn [com.arjuna.ats.arjuna] (transaction reaper worker 0) arjuna012108: checkedaction::check - atomic action 0:ffff0a00000b:2d6361a7:5435c20e:22 aborting 1 threads active! 00:06:37,184 warn [com.arjuna.ats.arjuna] (transaction reaper worker 0) arjuna012121: transactionreaper::docancellations worker thread[transaction reaper worker 0,5,main] canceled tx 0:ffff0a00000b:2d6361a7:5435c20e:22 00:06:37,193 error [org.jboss.as.ejb3] (default task-38) javax.ejb.ejbtransactionrolledbackexception: jbas011469: transaction required perform operation (either utilize transaction or extended persistence context) 00:06:37,193 error [org.jboss.as.ejb3.invocation] (default task-38) jbas014134: ejb invocation failed on component myitemdao method public java.lang.object model.dao.basedao.update(java.lang.object): javax.ejb.ejbtransactionrolledbackexception: jbas011469: transaction required perform operation (either utilize transaction or extended persistence context) @ org.jboss.as.ejb3.tx.cmttxinterceptor.handleincallertx(cmttxinterceptor.java:163) [wildfly-ejb3-8.1.0.cr2.jar:8.1.0.cr2] @ org.jboss.as.ejb3.tx.cmttxinterceptor.invokeincallertx(cmttxinterceptor.java:253) [wildfly-ejb3-8.1.0.cr2.jar:8.1.0.cr2] @ org.jboss.as.ejb3.tx.cmttxinterceptor.required(cmttxinterceptor.java:342) [wildfly-ejb3-8.1.0.cr2.jar:8.1.0.cr2] @ org.jboss.as.ejb3.tx.cmttxinterceptor.processinvocation(cmttxinterceptor.java:239) [wildfly-ejb3-8.1.0.cr2.jar:8.1.0.cr2] @ org.jboss.invocation.interceptorcontext.proceed(interceptorcontext.java:309)

, lot more warnings , errors. subsequent insert statements fail.

i've searched based on arjuna error code, answers i've found seem suggest database or table beingness locked due mysqldump. not case me - there no mysqldump in progress.

another suggestion i've seen number of mysql connections might beingness exhausted, i'm not sure how connections allocated jpa. have 2 tables beingness updated, , i'm merging , flushing each entity populated source info file.

my info file , processing looks this...

<!-- reference info file --> <!-- contains 1 x elem_1 record, approx 1000 elem_2 records, , approx 60,000 elem_3 records --> <!-- there varying number of elem_3 records per elem_2 record --> <elem_1 attr1="" attr2=""> <!-- create detached elem_1 entity --> <!-- persist elem_1 entity , flush elem_1 em --> <elem_2 attr1=""> <!-- create detached elem_2 entity --> <val_1>value..</val_1> <!-- add together value detached entity --> <val_2>value..</val_2> <!-- add together value detached entity --> <val_3>value..</val_3> <!-- add together value detached entity --> <!-- persist elem_2 entity , flush elem_2 em --> <elem_3 attr1="" attr2="" attr3=""> <!-- create detached elem_3 entity --> <child_1>value..</child_1> <!-- add together value detached entity --> <child_2>value..</child_2> <!-- add together value detached entity --> <child_3>value..</child_3> <!-- add together value detached entity --> </elem_3> <!-- persist elem_3 entity , flush elem_3 em --> <!-- repeat above elem_3 process elem_3 records --> <!-- phone call clear() on elem_3 em --> </elem_2> <!-- flush elem_2 em phone call clear() on elem_2 em --> <!-- repeat above elem_2 process elem_2 records --> </elem_1>

can suggest might causing please? more importantly, how resolve can load of data?

looks transaction manager has timeout transactions, , massive insert exceeds timeout, transaction reaper aborts long running transaction , next insert fails.

there's config setting increment timeout, imho you'd improve split massive insert smaller batches separate transactions.

mysql jpa eclipselink wildfly