Parse into Website - c#

I have a file (text or csv) that I generate that has a list of part numbers. I need to take this list, and download some spec sheets for these parts automatically and then print. Once on the website, I need to input the part number, then print the results. What's the best way to do this?
Okay everyone, here's what I was doing before, but it would take over an hour to process on a progress 4gl (version 9.1) database into a QAD environment v8.6e on a Unix Red-Hat server:
FNAME=`date +%y%m%d%H%M%S`
echo requiredmcpartno=$1 | lynx -accept_all_cookies -nolist -dump -post_data 75.144.##.###/specdata/specdata.asp 2>&1 | tee $FNAME | lp -d$2 >>/apps/proedi/####/ftp/log/brownart.log
grep "Unit of Issue" $FNAME | cut --delimiter=: --fields=2 | awk '{print $1}'
grep -q "PACKAGING SPEC IS OBSOLETE FOR THIS PART NUMBER" "$FNAME"
if [ $? -eq 0 ]; then
echo 0
echo nopic
exit
fi
cd /apps/proedi/####/ftp/ftpscripts
rm -fr 184.168.##.###/ 75.144.##.###/ www.google-analytics.com/ 2>&1 >>/apps/proedi/####/ftp/log/brownart.log
wget -p -m -k -K -E -H --cookies=on --post-data="requiredmcpartno=$1" 75.144.##.###/specdata/specdata.asp >/dev/null 2>&1
/apps/proedi/####/ftp/ftpscripts/printspec.sh $1 $2 >>/tmp/printspec.log 2>&1
cat /apps/proedi/####/ftp/ftpscripts/"$1".pt
rm -f /apps/proedi/####/ftp/ftpscripts/"$1".pt
>>/apps/proedi/####/ftp/log/brownart.log
rm $FNAME 2>&1 >>/apps/proedi/ford/ftp/log/brownart.log
Then printspec.sh script:
file=184.168.70.174/partandpackagingphotos/PhotoDetailsSpecdata.aspx\?p\=$1.html
if [ ! -f "$file" ]; then
echo nopic >/apps/proedi/ford/ftp/ftpscripts/"$1".pt
exit
fi
grep -q "No Pictures Available for this Part Number" "$file"
if [ $? -eq 0 ]; then
echo nopic >/apps/proedi/ford/ftp/ftpscripts/"$1".pt
exit
fi
html2ps -i .7 184.168.70.174/partandpackagingphotos/PhotoDetailsSpecdata.aspx\?p\=$1.html | lp d$2 -s
echo picfnd >/apps/proedi/ford/ftp/ftpscripts/"$1".pt
The wget command takes way to long to process in Unix. Our customer may send us a conveyance file with 150-200 parts 8-9 times a day, and we need to download all of the pictures associated with each part every time we receive parts.
I was thinking of just making a flat file(text or csv), then have the user run a batch file on their windows computer to connect to the unix server, and download the file to their computer. After this, then have either the same batch job, or an excel template or something on the windows side download the pictures and print the spec sheets to their default printer.
Sorry for not posting all of this initially.

The first thing that I would try is to break the process into two or more independent pieces and run them in parallel. The scripts above appear to take a part number as a parameter. I would guess that whatever is feeding them the part number is working from a list (the "conveyance file"?) That list would be the obvious place to make the split.
If you do it in such a way that the number of concurrent processes is configurable it should be simple to find the "sweet spot". Supposing that the list of parts to be downloaded is in a table called "part" with fields "needsDownload" and "partNum". For the sake of simplicity we will assume that partNum is an integer and that the actual part numbers needing download are randomly distributed. If you are driving this process with Progress 4GL code you might write a control program something like this:
/* control.p
*
* to run two "threads":
*
* _progres -b dbName -p control.p -param "1,2" > control.1.log 2>&1 & # 1 of 2
* _progres -b dbName -p control.p -param "2,2" > control.2.log 2>&1 & # 2 of 2
*
*
*/
define variable myThread as integer no-undo.
define variable numThreads as integer no-undo.
myThread = integer( entry( 1, session:parameter )) - 1. /* this just allows the "1 of 2" stuff to be more "human sensible" */
numThreads = integer( entry( 2, session:parameter )).
for each part exclusive-lock where needsDownload = true and (( partNum modulo numThreads ) = myThread ):
os-command value( "getpart.sh " + string( partNum )).
needsDownload = false.
end.
Of course the problem might be that the external system is too slow. No amount of programming on your end will fix that.

Related

Jenkins build blocks because of a windows form

We have a .net application where it checks whether the build is on release mode and open up a simple windows form to input the version as a pre build event. I made this form to automatically close in 10 seconds if the user does not give an input. But unfortunately, in Jenkins, the build gets stuck on this step without going forward. So my guess was since Jenkins runs on command line it waits until the user input for continue. But even when I add automatically close the form it does not continue. Is there a way to build this job without UI blocking Jenkins?
You are not using Jenkins in the optimal way. Here are a few tips to help you out:
Get rid of your windows form to increment version
Add a CommonAssemblyInfo.cs in your visual studio solution with an initial version number
Force Jenkins to increment the version automatically [described below]
Commit the file by jenkins using git publisher or using svn.exe with commit flag
Reading Version number using powershell:
param([string]$assemblyInfoPath, [string]$workSpace)
$contents = [System.IO.File]::ReadAllText($assemblyInfoPath)
$versionString = [RegEx]::Match($contents,"(AssemblyFileVersion\("")(?:\d+\.\d+\.\d+\.\d+)(""\))")
Write-Host ("AssemblyFileVersion: " +$versionString)
$version = gc $assemblyInfoPath | select-string -pattern "AssemblyVersion"
$version -match '^\[assembly: AssemblyVersion\(\"(?<major>[0-9]+)\.(?<minor>[0-9]+)\.(?<revision>[0-9]+)\.(?<build>[0-9]+)\"\)\]'
$BuildVersionNumber = $matches["major"]+"."+$matches["minor"]+"."+$matches["revision"]+"."+$matches["build"]
Write-Host ("WorkSpace: " + $env:WORKSPACE.ToString()+"\version.txt")
#[Environment]::SetEnvironmentVariable("BUILD_NUMBER", $BuildVersionNumber, "Machine")
$path = $env:WORKSPACE.ToString() + "\version.txt"
$BuildVersionNumber | out-file -encoding ASCII -filepath $path
Increment Version using powershell:
#
# This script will increment the build number in an AssemblyInfo.cs file
#
param([string]$assemblyInfoPath, [string]$workSpace)
$contents = [System.IO.File]::ReadAllText($assemblyInfoPath)
$versionString = [RegEx]::Match($contents,"(AssemblyFileVersion\("")(?:\d+\.\d+\.\d+\.\d+)(""\))")
Write-Host ("AssemblyFileVersion: " +$versionString)
#Parse out the current build number from the AssemblyFileVersion
$currentBuild = [RegEx]::Match($versionString,"(\.)(\d+)(""\))").Groups[2]
Write-Host ("Current Build: " + $currentBuild.Value)
#Increment the build number
$newBuild= [int]$currentBuild.Value + 1
Write-Host ("New Build: " + $newBuild)
#update AssemblyFileVersion and AssemblyVersion, then write to file
Write-Host ("Setting version in assembly info file ")
$contents = [RegEx]::Replace($contents, "(AssemblyVersion\(""\d+\.\d+\.\d+\.)(?:\d+)(""\))", ("`${1}" + $newBuild.ToString() + "`${2}"))
$contents = [RegEx]::Replace($contents, "(AssemblyFileVersion\(""\d+\.\d+\.\d+\.)(?:\d+)(""\))", ("`${1}" + $newBuild.ToString() + "`${2}"))
[System.IO.File]::WriteAllText($assemblyInfoPath, $contents)
$version = gc $assemblyInfoPath | select-string -pattern "AssemblyVersion"
$version -match '^\[assembly: AssemblyVersion\(\"(?<major>[0-9]+)\.(?<minor>[0-9]+)\.(?<revision>[0-9]+)\.(?<build>[0-9]+)\"\)\]'
$BuildVersionNumber = $matches["major"]+"."+$matches["minor"]+"."+$matches["revision"]+"."+$matches["build"]
Write-Host ("WorkSpace: " + $env:WORKSPACE.ToString()+"\version.txt")
#[Environment]::SetEnvironmentVariable("BUILD_NUMBER", $BuildVersionNumber, "Machine")
$path = $env:WORKSPACE.ToString() + "\version.txt"
$BuildVersionNumber | out-file -encoding ASCII -filepath $path
Usage in Jenkins:
Version Format in CommonAssembly: 1.0.0.0
After incrementing: 1.0.0.1
As Tom mentioned in the comments, you should look add the option to start your forms application with a parameter that indicates that the main form should not be shown and some magic should happen. For example, check for a "/s" and let the application run silently if it is present:
MyWinformsApplication.exe /s
Also, as Tom mentioned, a console application can still open a window and this is really useful to have in Jenkins as you can then write insightful messages to the console which will be logged by Jenkins. You can always use these at a later stage to check if something went wrong.
As an additional note - if you add Console.WriteLine() to your WinForms application, Jenkins will pick the string up and add it to the console log.

command line arguments given to a C# program are parsed wrong

I wrote a little program, that has the following command line for example:
prog.exe -p -z "E:\temp.zip" -v "f:\" -r -o -s –c
prog.exe -p -z "E:\temp.zip" -v f:\ -r -o -s –c
prog.exe -p -z "E:\temp.zip" -v "f:\log" -r -o -s –c
This command line is being generated by another program that inserts the quotation marks around the file and path names automatically to prevent spaces being recognized as argument separators.
The first command line is then being parsed by the .Net framework as:
args {string[5]} string[]
[0] "-p" string
[1] "-z" string
[2] "f:\\temp.zip" string
[3] "-v" string
[4] "f:\" -r -o -s -c" string
But it should be ( the result from the second command line):
args {string[9]} string[]
[0] "-p" string
[1] "-z" string
[2] "f:\\temp.zip" string
[3] "-v" string
[4] "f:\\" string
[5] "-r" string
[6] "-o" string
[7] "-s" string
[8] "-c" string
A possible solution would be to check the file and path names in the calling application if they contain spaces and only then append the quotation marks around the names.
But is there another solution?
I created a little batch file that would display the command line parameters. With your first command line, I get
1 -p
2 -z
3 "E:\temp.zip"
4 -v
5 "f:\"
6 -r
7 -o
8 -s
9 –c
as it should. If .NET parses it differently, that would be a major bug. Just to check, I created a test console app. The results were:
0 -p
1 -z
2 E:\temp.zip
3 -v
4 f:" -r -o -s -c
I didn't think that was possible! So I researched a little and I got to this link: Everyone quotes command line arguments the wrong way which explains why the command line parser is essentially flawed. (I know it's for C++, but it applies) which led me to the rules of parsing for command line parameters which says \" is interpreted as an escaped quote, as opposed to how the operating system sees it (which is gross).
Conclusion: if you want to fix the command line, you need to escape the slash before a quote so instead of "f:\" you need "f:\\". The other solution is to use Environment.CommandLine which gets you the entire command line, executable included, and you can parse it yourself. More on this here: Getting raw (unsplit) command line in .NET.
Just for completeness' sake, I'll throw this in: Split string containing command-line parameters into string[] in C#, which discusses how to split a command line string into parameters using system functions.
After some more research, I realized that there are several standards
of command line argument parsing, depending on compiler and operating
system and its version, and that the only way to be consistent is to
parse the raw command line yourself.
For example, using CommandLineToArgvW on Environment.CommandLine replicated exactly a call to Environment.GetCommandLineArgs() which returns the exact same thing as the args array given to the Main method.
I found a very detailed page about this problem: http://daviddeley.com/autohotkey/parameters/parameters.htm which I believe is the definitive answer to the general question of command line parsing.

How to use Regex in batch file

I am using batch file in tfs build step.
I wanted to parse a text file from batch script.
File content is something like this
-------------------------------------------------------------------------------
Changeset: 267406
User: myname
Date: Wednesday, August 12, 2015 5:33:28 PM
Comment:
TFS Changes
Items:
edit $/dummyPath/uDClient/PushVersion.bat
add $/dummyPath/uDClient/request.json
-------------------------------------------------------------------------------
Changeset: 267356
User: username
Date: Wednesday, August 12, 2015 12:16:58 PM
Comment:
Initial check-in
Items:
edit $/dummyPath/UserControls/Welcome.ascx
edit $/dummyPath/UserControls/Welcome.ascx.cs
edit $/dummyPath/UserControls/Welcome.ascx.designer.cs
deleted $/dummyPath/UserControls/p1.ascx
Required out put will be
$/dummyPath/uDClient/PushVersion.bat
$/dummyPath/uDClient/request.json
$/dummyPath/UserControls/Welcome.ascx
$/dummyPath/UserControls/Welcome.ascx.cs
$/dummyPath/UserControls/Welcome.ascx.designer.cs
$/dummyPath/UserControls/p1.ascx
I wanted to do this parsing in batch script, if it is not possible then i'll go for C# Regex through utility application.
So can anybody suggest me how can i do this in batch script file
I am using below command to get required output
for /f "tokens=2" %i in ('FINDSTR /C:^edit p.txt') do echo %i
but i have to run same commond three times (for add,delete also)
in c# regex we can specify list words to match
*(add|delete|edit)\s*
How i can use alternate matching words in findstr.
This code with your text in file.txt will print the output below it:
#echo off
for /f "tokens=1,* delims=$" %%a in ('find "/" ^<"file.txt"') do echo $%%b
output:
$/dummyPath/uDClient/PushVersion.bat
$/dummyPath/uDClient/request.json
$/dummyPath/UserControls/Welcome.ascx
$/dummyPath/UserControls/Welcome.ascx.cs
$/dummyPath/UserControls/Welcome.ascx.designer.cs
$/dummyPath/UserControls/p1.ascx
You can try to use a Power Shell script to do it.
Here is a powershell script which produces the desired output on your specified input:
select-string -Path 'C:\source\source.txt' -Pattern '\$.*' -AllMatches | % { $_.Matches } | % { $_.Value }

how to set dynamic arguments for a process?

In bash
I run the following kill commands (it's just a kill with an expression which return the ID of a process)
kill $(ps -ef | grep '[m]atchbox-panel --titlebar --start-applets showdesktop,windowselector' | cut -f8 -d' ')
which return something like
kill 800
When I try to run this in C#
Process.Start("kill", "$(ps -ef | grep '[m]atchbox-panel --titlebar --start-applets showdesktop,windowselector' | cut -f8 -d' ')");
I get a fatal error.
Alternative
I could run all the commands one by one until I have the number and set a process with "kill" and the "number", but it seems a lot of trouble for what it does.
$(..) is not evaluated in a Process.Start argument, that is a feature of bash. You can split your command in two lines, or try pkill.

Batch to copy files to another folder with breaks

I need a script, prefferably a windows batch or C# to do as following:
Show a prompt that first ask for the source folder,
then it should ask for the destination folder. At last, it shall ask how many files it should copy to the destination, from the source.
// We talk about aprox 100.000 files and they can be moved in random order.
After the process has been run, the program shall make a break of 10 minutes, then loop the process it was told to earlier, by previous answers to the prompt.
I've tried a little, but haven't found a solution. As far as i can see, XCOPY is unable to work around all these criterias.
Thanks in advance,
Mark
RoboCopy (the See also section might interest you as well) or
(more recent:) RichCopy (download)
You can use something like this:
string source = Console.ReadLine();
string destination = Console.ReadLine();
int numberOfFilesToCopy = int.Parse(Console.ReadLine());
DirectoryInfo di = new DirectoryInfo(source);
var files = di.GetFiles();
for(i=0;i < math.Max(files.Length, numberOfFilesToCopy);i++)
{
files[i].CopyTo(destination);
}
In C# using System.IO.File.Copy(sourceFileName,destFileName) followed by a System.IO.File.Delete(path) will do the "move" for you. You can create a simple console app that takes in the information you need and then does the work.
Have a look at the docs for System.IO.File for more info on File operations.
I'm not sure this fulfills all your requirements, but it may be useful to have a look at robocopy (robocopy /? in the command line).
Don't think I missed anything =D
#ECHO OFF
::User Prompts
SET /p source=Source Folder? Use format DRIVE:\PATH\ :
SET /p destination=Destination Folder? Use format DRIVE:\PATH\ :
SET /p count=How many files to copy? :
::Setup the Batch file to schedule
DIR /B "%source%">>"%userprofile%\batchtemp\source.BAT"
SET batchfile=%userprofile%\batchtemp\source.BAT
ECHO SETLOCAL ENABLEDELAYEDEXPANSION>>"%batchfile%"
ECHO FOR /F "USEBACKQ tokens=*" %%A IN ("%batchfile%") DO ( >>"%batchfile%"
ECHO COPY /Y "%%~fA" "%destination%\%%~nxA">>"%batchfile%"
ECHO SET /a count=!count!-1>>"%batchfile%"
ECHO IF %count% EQU 0 GOTO CLEANUP>>"%batchfile%"
ECHO )>>"%batchfile%"
ECHO :CLEANUP>>"%batchfile%"
ECHO ENDLOCAL>>"%batchfile%"
::Setup the scheduled task based on a future time in minutes.
REM Given that the job will run on the same day not overlapping a 24 hour day
FOR /F "tokens=1-3 delims=: " %%F IN ('time /t') DO (
SET hours=%%F
SET minutes=%%G
)
FOR /F "tokens=1-4 delims=/ " %%F IN ('date /t') DO (
SET day=%%F
SET thedate=%%G/%%H/%%I
)
SET /a minutes=%minutes%+10
IF %minutes% GRT 60 SET /a minutes=%minutes%-60 & SET /a hours=%hours%+1
SCHTASKS /Create /TR "%batchfile%" /ST %hours%:%minutes%:00 /MO ONCE /D %day% /SD "%thedate%" /ED "%thedate%" /TN "Copy Files"

Categories

Resources