[pmwiki-users] PmWiki cookbook recipe for Anti-virus scanning

Thu Jun 14 12:57:57 CDT 2012

I can't do it right now and I have a lot of recipes to fix and finish,
but it doesn't look so complex to integrate the API provided by the
the site virustotal with a recipe for pmwiki.

Instead of uploading files for scaning one can just send the urls of
the files to be scanned - that would avoid timeouts - the process is
slow on the virus total site as all files are queued and have a lower
priority than the files sent directly to the virus total website
trough the form on their site, but hey, it is a free service and also
helps the virus companies to identify more virus and grow their
databases.

The way I see it - keep in mind I'm not even a good amateur programmer
- an action could be created to select the files to be sent for
scanning, urls are sent  back to pmwiki, pmwiki formats the urls to be
sent to virustotal trough api, urls are sent, a key for the specific
report is kept, next time the action is used, pmwiki checks for report
automagically, if a report is not found, let's wait for another round,
if a report is found, it gets parsed, info about each url sent will be
placed next to files they represent, the report about virus scanning
is made, include options to delete or put files to quarentine.

You'll need a key to use the API, but it is free and provided at the site.

Also, there are some limits on the amount of urls you can send, but I
belive that picking the files individually for checking from a list,
the limits will never be reached.

I have done just a quick read on the API, but anyone can find more
information on it trough the links below.

Virus total website : http://www.virustotal.com
API docs : https://www.virustotal.com/documentation/public-api/
PHP script for acessing virustotal API :
http://www.ab-weblog.com/en/php-script-for-accessing-virustotal-api-version-2-0/

It would be a great addition to the cookbook.

CarlosAB

----

On 5/25/12, a.sonderhoff at gassi-tv.de <a.sonderhoff at gassi-tv.de> wrote:
> Long answer: Given the amount of interfacing and server side issues
> (especially file transfer), I don’t think that such a script could be easily
> written in a mere couple of hours. I am not even close to being a real
> programmer though, so maybe someone can figure out an easy way to do this in
> just a couple of hours. It stands to reason though, since this is an issue
> almost every collaboration centered web application has, that someone would
> have already done it, if it were that simple.
>
> Short answer: If you really want to have this functionality, you will
> probably have to hire someone to develop this recipe for you.
>
> -josh
>
>
> On May/24, 2012, at 2318 , Al Louis Ripskis wrote:
>
>> May 20, 2012 4:46 PM a.sonderhoff at gassi-tv.de (Josh) wrote:
>>> writing a PmWiki cookbook recipe, that submits uploaded files to an
>>> online virus >scanner.
>>
>> Would this be a relatively simple challenge that somebody who knows what
>> they are doing could dash off in a couple of hours?
>> Maybe we could make a game of it: who could be the first to come up with a
>> simple, functional recipe. It might be a welcome addition to PmWiki.
>> Al
>>
>>
>> -----Original Message-----
>>> From: a.sonderhoff at gassi-tv.de
>>> Sent: May 20, 2012 4:46 PM
>>> To: Al Louis Ripskis <ripskis at sprynet.com>
>>> Cc: PmWiki Users <pmwiki-users at pmichaud.com>
>>> Subject: Re: [pmwiki-users] Anti-virus scanning & PmWiki
>>>
>>> Antivirus scanning is something you really don’t want a php script to
>>> perform. You would face script timeouts, slow performance, memory issues,
>>> etc. So unless you want to prepackage PmWiki with a server OS bundle and
>>> throw in apache, clamav, php-clamav, etc, it is not doable. There are far
>>> to many different web server and operating system configurations.
>>>
>>> Unless it would be done without involving the local filesystem, e. g.
>>> writing a PmWiki cookbook recipe, that submits uploaded files to an
>>> online virus scanner. You would still have to deal with various timeout
>>> and timing issues, but this is actually something that could work. Given
>>> the amount of feedback to this thread, it’s probably not very high up on
>>> the priority list though.
>>>
>>> Josh
>>>
>>>
>>> On May/20, 2012, at 1937 , Al Louis Ripskis wrote:
>>>
>>>> May 19, 2012 4:22 PM a.sonderhoff at gassi-tv.de wrote:
>>>> For all those interested in FOSS-based approaches to virus scanning for
>>>> web app servers, this is an article I found while researching whether
>>>> there was an easy solution to Al’s request:
>>>>> <http://www.techspot.co.in/2009/03/file-upload-and-virus-scanners.html>
>>>>
>>>> The take home message that I got from examining the above site, et al.
>>>> is keep it simple: that it can be a "nightmare" and very expensive if
>>>> one gets into meta scanning and collaborative antivirus programs.
>>>> Instead use one program, such as AVG, and integrate it into PmWiki. The
>>>> question is, is it doable architecturally?
>>>> Al
>>>>
>>>> -----Original Message-----
>>>>> From: a.sonderhoff at gassi-tv.de
>>>>> Sent: May 19, 2012 4:22 PM
>>>>> To: tamouse mailing lists <tamouse.lists at gmail.com>
>>>>> Cc: PmWiki Users <pmwiki-users at pmichaud.com>
>>>>> Subject: Re: [pmwiki-users] Solved: How do you creating one,
>>>>> wiki-wide,	Upload Directory on Local Site, not remote site?
>>>>>
>>>>> For all those interested in FOSS-based approaches to virus scanning for
>>>>> web app servers, this is an article I found while researching whether
>>>>> there was an easy solution to Al’s request:
>>>>> <http://www.techspot.co.in/2009/03/file-upload-and-virus-scanners.html>
>>>>>
>>>>>
>>>>> On May/19, 2012, at 0019 , tamouse mailing lists wrote:
>>>>>
>>>>>> Hi, Al,
>>>>>>
>>>>>> This can be automated, but it's not exactly a one or two-line script.
>>>>>> I think others and myself discussed this a bit in your last thread
>>>>>> about virus checking. If you aren't up for writing it yourself,
>>>>>> perhaps you might engage the services of someone who can do it for
>>>>>> you.
>>>>>>
>>>>>> On Fri, May 18, 2012 at 3:46 PM, Al Louis Ripskis
>>>>>> <ripskis at sprynet.com> wrote:
>>>>>>> Many thanks to Josh, Gilles, Tamouse and JDD for helping me resolve
>>>>>>> this conundrum.
>>>>>>> Since PmWiki architecture apparently doesn't permit to automate this
>>>>>>> process, here is how I'm dealing with it:
>>>>>>> I manually (via FileZilla) transfer all the files From the
>>>>>>> PmWiki-wide uploads directory to my local
>>>>>>> C:\public_html\UnTestedUploads directory, where I can use my AVG
>>>>>>> anti-virus program to check out and delete the infected files, then
>>>>>>> process the uninfected files as necessary.
>>>>>>> It would be nice to be able to do this automatically, but c'est la
>>>>>>> vie!
>>>>>>> Thanks to all,
>>>>>>> Al
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>>> From: a.sonderhoff at gassi-tv.de
>>>>>>>> Sent: May 17, 2012 3:21 PM
>>>>>>>> To: Al Louis Ripskis <ripskis at sprynet.com>
>>>>>>>> Cc: pmwiki-users <pmwiki-users at pmichaud.com>
>>>>>>>> Subject: Re: [pmwiki-users] How do you creating one, wiki-wide,
>>>>>>>> Upload Directory on Local Site, not remote site?
>>>>>>>>
>>>>>>>> Dear Al,
>>>>>>>>
>>>>>>>> obviously this can’t work. your remote config.php will only affect
>>>>>>>> the remote settings and therefore only manipulate the remote file
>>>>>>>> system.  to achieve what you want to do, you will…
>>>>>>>>
>>>>>>>> 1) …either have to mirror the site using something like rsync’ing
>>>>>>>> with the help of a cron job (and have all the problems discussed in
>>>>>>>> an earlier desktop virus scan software thread in this list, which
>>>>>>>> you will probably find in the mailing list archives)
>>>>>>>>
>>>>>>>> 2) …or write some script that does what you need it to do (basically
>>>>>>>> find a way to interface the local and the remote sites).
>>>>>>>>
>>>>>>>>
>>>>>>>> assuming your primary goal is not to move pages from one pmwiki
>>>>>>>> instance to another and have them cross-reference/load their pages,
>>>>>>>> but rather have a way to scan your uploaded files for
>>>>>>>> malware/viruses/etc, you should probably do one of those things:
>>>>>>>>
>>>>>>>> a) if you need to run the scan from your workstation, rather than
>>>>>>>> running it on the server itself, the easiest way would be to mount
>>>>>>>> the server directory as a smb share on your local machine (if your
>>>>>>>> remote and local sites are not on the same network, you will
>>>>>>>> probably have to use a vpn tunnel for that). to explain how to do
>>>>>>>> that is kind of out of the scope of this mailing list, as it is not
>>>>>>>> a PmWiki specific question, but more of a general server
>>>>>>>> administration/setup task.
>>>>>>>>
>>>>>>>> b) you could consider installing a virus scanner (e. g.
>>>>>>>> clamav-server) on your remote site’s server and have it watch your
>>>>>>>> uploads folder and subdirs. this would definitely be a better
>>>>>>>> solution, but users still won’t get feedback why their file has
>>>>>>>> suddenly disappeared. disappearing files is still better than
>>>>>>>> serving malware. and if you have to much spare time on your hand you
>>>>>>>> could even come up with a solution how to have clamav-server tell
>>>>>>>> pmwiki which files had to be removed.
>>>>>>>>
>>>>>>>> c) you could come up with a new way for pmwiki to upload files (e.
>>>>>>>> g. write a new uploader) which automatically triggers your server’s
>>>>>>>> antivirus software and only successfully moves the file into place
>>>>>>>> after it has been cleared by the av software.
>>>>>>>>
>>>>>>>>
>>>>>>>> again, this topic is not really related to PmWiki in the first
>>>>>>>> place, since it’s a problem basically every content management
>>>>>>>> system has. you might have better luck searching the web for a more
>>>>>>>> general solution to this problem, or maybe ask this question in a
>>>>>>>> more generalized way on stackoverflow.
>>>>>>>>
>>>>>>>> Josh
>>>>>>>>
>>>>>>>> On May/17, 2012, at 1957 , Al Louis Ripskis wrote:
>>>>>>>>
>>>>>>>>> May 17, 2012 8:57 AM a.sonderhoff at gassi-tv.de; Josh wrote:
>>>>>>>>> $UploadDir = "C:/public_html/pmwiki/uploads";
>>>>>>>>> $UploadUrlFmt =
>>>>>>>>> "http://www.politicaltransformation.org/htdocs/pmwiki/uploads";
>>>>>>>>>> Did you put these lines in your local or remote site’s
>>>>>>>>>> local/config.php?
>>>>>>>>>> Can you clarify what your local and remote sites actually are?
>>>>>>>>> The remote site is my politicaltransformation.org PmWiki and that's
>>>>>>>>> the config.php file that I inserted the above two lines of script.
>>>>>>>>> Now my local site is the computer I'm operating from that has
>>>>>>>>> Windows XP Home OS.
>>>>>>>>> Thanks again,
>>>>>>>>> Al
>>>>>>>>>
>>>>>>>>>> On May/17, 2012, at 1414 , Al Louis Ripskis wrote:
>>>>>>>>>>
>>>>>>>>>>> May 16, 2012 10:11 PM Tamouse wrote: Do I understand this
>>>>>>>>>>> correctly.Let me know if that's the situation so I can respond:
>>>>>>>>>>>> a) politicaltransformation.org is hosted on a remote site? YES
>>>>>>>>>>>> b) you are expecting the remote site to understand a path on
>>>>>>>>>>>> your local machine? YES
>>>>>>>>>>>> c) you are further expecting the wiki software to deliver a file
>>>>>>>>>>>> to a user by cross-loading it from your local machine? NO.
>>>>>>>>>>>> Deliver the user uploaded files to Local Upload directory so
>>>>>>>>>>>> they can be checked out by my AVG virus program for viruses,
>>>>>>>>>>>> etc..If this is not possible, is there another way to check out
>>>>>>>>>>>> automatically whether the uploaded files are infected?
>>>>>>>>>>> Thanks very much,
>>>>>>>>>>> Al
>>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: tamouse mailing lists
>>>>>>>>>>>> Sent: May 16, 2012 10:11 PM
>>>>>>>>>>>> To: Al Louis Ripskis
>>>>>>>>>>>> Cc: pmwiki-users
>>>>>>>>>>>> Subject: Re: [pmwiki-users] How do you creating one, wiki-wide,
>>>>>>>>>>>> Upload Directory on Local Site, not remote site?
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, May 16, 2012 at 1:43 PM, Al Louis Ripskis wrote:
>>>>>>>>>>>>> According to PmWiki Upload Administration instructions all you
>>>>>>>>>>>>> suppose to do
>>>>>>>>>>>>> is put this or equivalent in you Config file::
>>>>>>>>>>>>>
>>>>>>>>>>>>> $UploadDir = "/home/foobar/public_html/uploads";
>>>>>>>>>>>>> $UploadUrlFmt = "http://example.com/~foobar/uploads";
>>>>>>>>>>>>>
>>>>>>>>>>>>> When I translate the above to my Windows XP OS needs, I come
>>>>>>>>>>>>> with this:
>>>>>>>>>>>>>
>>>>>>>>>>>>> $UploadDir = "C:/public_html/pmwiki/uploads";
>>>>>>>>>>>>> $UploadUrlFmt =
>>>>>>>>>>>>> "http://www.politicaltransformation.org/htdocs/pmwiki/uploads";
>>>>>>>>>>>>>
>>>>>>>>>>>>> But all this does is create another upload directory on the
>>>>>>>>>>>>> remote site.
>>>>>>>>>>>>> I would like to create one, site-wide directory on the local
>>>>>>>>>>>>> site.
>>>>>>>>>>>>> How do I do that?
>>>>>>>>>>>>
>>>>>>>>>>>> Do I understand this correctly:
>>>>>>>>>>>>
>>>>>>>>>>>> a) politicaltransformation.org is hosted on a remote site?
>>>>>>>>>>>> b) you are expecting the remote site to understand a path on
>>>>>>>>>>>> your local machine?
>>>>>>>>>>>> c) you are further expecting the wiki software to deliver a file
>>>>>>>>>>>> to a
>>>>>>>>>>>> user by cross-loading it from your local machine?
>>>>>>>>>>>>
>>>>>>>>>>>> Let me know if that's the situation so I can respond.
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> pmwiki-users mailing list
>>>>>>>>>>> pmwiki-users at pmichaud.com
>>>>>>>>>>> http://www.pmichaud.com/mailman/listinfo/pmwiki-users
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> pmwiki-users mailing list
>>>>>> pmwiki-users at pmichaud.com
>>>>>> http://www.pmichaud.com/mailman/listinfo/pmwiki-users
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> pmwiki-users mailing list
>>>>> pmwiki-users at pmichaud.com
>>>>> http://www.pmichaud.com/mailman/listinfo/pmwiki-users
>>>>
>>>
>>
>
>
> _______________________________________________
> pmwiki-users mailing list
> pmwiki-users at pmichaud.com
> http://www.pmichaud.com/mailman/listinfo/pmwiki-users
>