You are here: Foswiki>Tasks Web>Component>SolrPlugin>Item11816 (29 Aug 2013, MichaelDaum)Edit Attach

Item11816: SolrIndex fails with Broken Pipe Error

pencil
Priority: Urgent
Current State: No Action Required
Released In: n/a
Target Release: n/a
Applies To: Extension
Component: MetaCommentPlugin, SolrPlugin
Branches: trunk
Reported By: OliverSchaub
Waiting For: Main.MichaelDaum
Last Change By: MichaelDaum
We have a big Problem that rather big Webs can't get indexed anymore. The Indexer eventually fails with a "Broken Pipe" Error: 

write(8, "\27\3\1\0\220Y\314\263\0$U-t\333\367!\344\361\232\20\343X\251\3456\371\331\333\377\222<\313\nD\320VfA\245\212n\20>\215\213\320\227\377W$\220\304\\\270,\245\373\nJZ\372A\315\322\2410\24\333\231C\332\346\217\331JW\1\221\275,\246\1778\n\17Z\\x\332\373\2350\2637\335\353#", 149) = -1 EPIPE (Broken pipe)
--- SIGPIPE (Broken pipe) @ 0 (0) ---

Looking through the Logfiles of the Wiki-Server and the Tomcat which runs Solr, we found an Error-Message which might (or might not) be part of the Problem:

Wiki-Log: 
ERROR: 400 Bad Request: Apache Tomcat/6.0.29 - Error report 
HTTP Status 400 - ERROR:unknown field 'collection'
type Status report
message ERROR:unknown field 'collection'
description The request sent by the client was syntactically incorrect (ERROR:unknown field 'collection').
Apache Tomcat/6.0.29
 at /path/to/foswiki115/lib/WebService/Solr.pm line 180

Tomcat-Log: 
SEVERE: org.apache.solr.common.SolrException: ERROR:unknown field collection
        at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:289)
        at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60)
        at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:139)
        at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69)
        at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
        at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:563)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:619)

This has been mentioned on the Foswiki IRC as well: http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2012-01-16,Mon&sel=169#l165

-- OliverSchaub - 04 May 2012

What does "big webs" mean in numbers?

That aside it seems as if your solr isn't able to index a single topic without a collection field in your schema.xml.

Please check your solr's schema.xml with the one coming with the plugin.

-- MichaelDaum - 07 May 2012

I compared the schema.xml from our installation with the latest from the Plugin. They are identical.

The Size of our "larger" webs range from 600+ to 3000+ Topics (plus attachments)

-- OliverSchaub - 09 May 2012

Can you attach your schema.xml file, just to make sure?

-- MichaelDaum - 09 May 2012

There you go...

-- OliverSchaub - 09 May 2012

... and this one is outdated. If you are using SolrPlugin from trunk, then you definitely need http://svn.foswiki.org/trunk/SolrPlugin/solr/multicore/conf/schema.xml, which is in version 1.4. Yours is 1.2.

On the other hand, MetaCommentPlugin is already released (i.e. non-trunk), but does already make use of the collection field of SolrPlugin from trunk. That's no good. Try to disable this using: 

--- lib/Foswiki/Plugins/MetaCommentPlugin/Core.pm       (revision 14783)
+++ lib/Foswiki/Plugins/MetaCommentPlugin/Core.pm       (working copy)
@@ -673,7 +673,7 @@
     my $commentDoc = $indexer->newDocument();
     $commentDoc->add_fields(
       'id' => $id,
-      'collection' => $collection,
+#      'collection' => $collection,
       'language' => $language,
       'name' => $comment->{name},
       'type' => 'comment',

-- MichaelDaum - 10 May 2012

No, we don't use Solr from Trunk. We have version 1.10 installed and use the schema.xml from the .zip Attachment.

But we do have MetaComment installed. I'll comment this line out and have an eye on the Index-Logfiles.

-- OliverSchaub - 10 May 2012

There's another dependency on an unreleased change in SolrPlugin. Try disabling container_title for now. Below patch supersedes the above one: 

Index: lib/Foswiki/Plugins/MetaCommentPlugin/Core.pm
===================================================================
--- lib/Foswiki/Plugins/MetaCommentPlugin/Core.pm       (revision 14847)
+++ lib/Foswiki/Plugins/MetaCommentPlugin/Core.pm       (working copy)
@@ -673,7 +673,7 @@
     my $commentDoc = $indexer->newDocument();
     $commentDoc->add_fields(
       'id' => $id,
-      'collection' => $collection,
+#      'collection' => $collection,
       'language' => $language,
       'name' => $comment->{name},
       'type' => 'comment',
@@ -690,7 +690,7 @@
       'state' => ($comment->{state}||'null'),
       'container_id' => $web.'.'.$topic,
       'container_url' => Foswiki::Func::getViewUrl($web, $topic),
-      'container_title' => $indexer->getTopicTitle($web, $topic, $meta),
+#      'container_title' => $indexer->getTopicTitle($web, $topic, $meta),
     );
     $doc->add_fields('catchall' => $title);
     $doc->add_fields('catchall' => $comment->{text});

-- MichaelDaum - 18 May 2012

With this patch, those particular erros seem to have disappeared.

-- OliverSchaub - 21 May 2012

This is disabled in MetaCommentPlugin 2.0. I will enable it again as soon as the next SolrPlugin is out.

-- MichaelDaum - 09 Jul 2012

Fixed in latest combo of MetaCommentPlugin & SolrPlugin

-- MichaelDaum - 13 Aug 2013

Unfortfunately, the error is still there: 

[root@wise07 tools]# ./solrindex topic=Sandbox_D1.MetaComment1
Indexing topic Sandbox_D1.MetaComment1
ERROR: 400 Bad Request: {"responseHeader":{"status":400,"QTime":2},"error":{"msg":"ERROR: [doc=Sandbox_D1.MetaComment1] unknown field 'collection'","code":400}}
 at /wiki/foswiki115_solr/lib/WebService/Solr.pm line 194
        WebService::Solr::_send_update('WebService::Solr=HASH(0x3752270)', '<add><doc><field name="contributor">AndreLichtsteiner</field>...') called at /wiki/foswiki115_solr/lib/WebService/Solr.pm line 79
        WebService::Solr::add('WebService::Solr=HASH(0x3752270)', 'WebService::Solr::Document=HASH(0x3a0a9f0)') called at /wiki/foswiki115_solr/lib/Foswiki/Plugins/SolrPlugin/Index.pm line 822
        Foswiki::Plugins::SolrPlugin::Index::add('Foswiki::Plugins::SolrPlugin::Index=HASH(0x369c300)', 'WebService::Solr::Document=HASH(0x3a0a9f0)') called at /wiki/foswiki115_solr/lib/Foswiki/Plugins/SolrPlugin/Index.pm line 587
        Foswiki::Plugins::SolrPlugin::Index::__ANON__() called at /usr/share/perl5/Error.pm line 415
        eval {...} called at /usr/share/perl5/Error.pm line 407
        Error::subs::try('CODE(0x3a97678)', 'HASH(0x3a97660)') called at /wiki/foswiki115_solr/lib/Foswiki/Plugins/SolrPlugin/Index.pm line 592
        Foswiki::Plugins::SolrPlugin::Index::indexTopic('Foswiki::Plugins::SolrPlugin::Index=HASH(0x369c300)', 'Sandbox_D1', 'MetaComment1', undef, undef) called at /wiki/foswiki115_solr/lib/Foswiki/Plugins/SolrPlugin/Index.pm line 268
        Foswiki::Plugins::SolrPlugin::Index::updateTopic('Foswiki::Plugins::SolrPlugin::Index=HASH(0x369c300)', 'Sandbox_D1', 'Sandbox_D1.MetaComment1') called at /wiki/foswiki115_solr/lib/Foswiki/Plugins/SolrPlugin/Index.pm line 112
        Foswiki::Plugins::SolrPlugin::Index::__ANON__() called at /usr/share/perl5/Error.pm line 415
        eval {...} called at /usr/share/perl5/Error.pm line 407
        Error::subs::try('CODE(0x3a01f38)', 'HASH(0x3a0a720)') called at /wiki/foswiki115_solr/lib/Foswiki/Plugins/SolrPlugin/Index.pm line 131
        Foswiki::Plugins::SolrPlugin::Index::index('Foswiki::Plugins::SolrPlugin::Index=HASH(0x369c300)') called at /wiki/foswiki115_solr/lib/Foswiki/Plugins/SolrPlugin.pm line 183
        Foswiki::Plugins::SolrPlugin::indexCgi('Foswiki=HASH(0x13425a0)') called at /wiki/foswiki115_solr/lib/Foswiki/UI.pm line 318
        Foswiki::UI::__ANON__() called at /usr/share/perl5/Error.pm line 415
        eval {...} called at /usr/share/perl5/Error.pm line 407
        Error::subs::try('CODE(0x949228)', 'HASH(0x1342150)') called at /wiki/foswiki115_solr/lib/Foswiki/UI.pm line 441
        Foswiki::UI::_execute('Foswiki::Request=HASH(0xc0adf8)', 'CODE(0x1337908)', 'command_line', 1, 'solrindex', 1) called at /wiki/foswiki115_solr/lib/Foswiki/UI.pm line 276
        Foswiki::UI::handleRequest('Foswiki::Request=HASH(0xc0adf8)') called /wiki/foswiki115_solr/lib/Foswiki/Engine/CLI.pm line 53
        Foswiki::Engine::CLI::run('Foswiki::Engine::Legacy=HASH(0xc0a528)') called
Committing index
I've tried the above patch as well, but it doesn't work. Disregard, the patch works.

We're using the latest SolrPlugin (installed from configure) and the latest MetaCommentPlugin. On the Server, the schmema.xml is version 1.5.

-- OliverSchaub - 23 Aug 2013

Your schema.xml is outdated. It doesn't have a collection field. Please upgrade it. Then reindex. See http://trac.foswiki.org/browser/trunk/SolrPlugin/solr/multicore/conf/schema.xml#L1117

-- MichaelDaum - 23 Aug 2013

No. It DOES have the collection Field. The schema.xml attached here is the old one, the one I attached last year.

the schema.xml we use is version 1.5 right out of the SolrPlugin and is identical with the version from your Link.

-- OliverSchaub - 23 Aug 2013

If your solr tells you it doesn't know about collection then it presumably is using the wrong schema.xml? Or put the other way: how coulld it moan about an unknown collection field even though it was told about it in its schema.xml? So there's something seriously jammed in your specific setup.

-- MichaelDaum - 23 Aug 2013

I double-checked again. There is only ONE schema.xml on the whole Server, the one from the Plugin. It wouldn't work WITHOUT a schema.xml, correct? So If it was just the wrong path, solr wouldn't work at all.

We run Solr on a dedicated Tomcat-Server. There we have two Main Folders:
  • .../tomcat/webapps/
    • here is the solr.war File
  • .../opt/solr/multicore/
    • This is Solr Home.
    • here are the conf and data Folders.
    • The conf-Folder holds the schema.xml.

On the Wiki-Server, the latest Solr-plugin is installed. The folder/Filestructure is:
  • .../wikiroot/solr
    • etc
    • logs
    • multicore (holds schema.xml v 1.5 in the conf Folder)
  • .../wikiroot/tools
    • Holds all the Solr Scriptfiles

Do you see something particularly wrong with this setup?

-- OliverSchaub - 29 Aug 2013

Please compare the two schema.xml files on both servers:

  1. tomcat-server: ...opt/solr/multicore/conf/schema.xml
  2. wiki-server: ...wikiroot/solr/multicore/conf/schema.xml

If they are different then copy (2) to (1).

-- MichaelDaum - 29 Aug 2013
 

ItemTemplate edit

Summary SolrIndex fails with Broken Pipe Error
ReportedBy OliverSchaub
Codebase 1.1.5
SVN Range
AppliesTo Extension
Component MetaCommentPlugin, SolrPlugin
Priority Urgent
CurrentState No Action Required
WaitingFor MichaelDaum
Checkins MetaCommentPlugin:d5f91733003f
TargetRelease n/a
ReleasedIn n/a
CheckinsOnBranches trunk
trunkCheckins MetaCommentPlugin:d5f91733003f
Release01x01Checkins
I Attachment Action Size Date Who Comment
schema.xmlxml schema.xml manage 32 K 09 May 2012 - 13:33 OliverSchaub  
Edit  | Attach | Print version | History: r20 < r19 < r18 < r17 | Backlinks | View wiki text | Edit wiki text | More topic actions
Topic revision: r20 - 29 Aug 2013, MichaelDaum
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy