Develop Reference

Creating files with a specified name,path and format from a object list

i need to create a bunch of files on my machine,currently i have a list of objects of type FileInfo that holds the name, path and format of the files that i need to create.
What would be the best way to go about this?i´m thinking of using a foreach loop, but my list has over 700000 objects and i´m not sure if that´s the best option.
Here's the function that i'm using to populate the list with the objects:
public static List<FileInfo> ReturnFileFromDBInfo()
{
var result = new List<FileInfo>();
SqlConnection conn = new SqlConnection();
conn.ConnectionString = #"data source = MYPC\SQLEXPRESS; database = MYDB; integrated security = TRUE";
conn.Open();
var query = "SELECT a.partNumber, b.fullPath,a.fileType,a.baseID FROM drawings a ,bases_bases b WHERE a.baseID = b.id";
var cmd = new SqlCommand(query, conn);
SqlDataReader dataReader = cmd.ExecuteReader();
while (dataReader.Read())
{
result.Add ( new FileInfo()
{
partNumber = dataReader["partNumber"].ToString(),
path = dataReader["fullPath"].ToString(),
fileType = dataReader["fileType"].ToString(),
baseID = dataReader["baseID"].ToString(),
});
}
return result;
}
It´s just a simple query that retrieves the file names,path and formats from a sql-server DB

Creating a list of 700000 objects is going to be memory hungry and relatively slow. Instead of returning a List, I suggest you return IEnumerable, like this:
public static IEnumerable<FileInfo> ReturnFileFromDBInfo()
{
var result = new List<FileInfo>();
using (SqlConnection conn = new SqlConnection())
{
conn.ConnectionString = #"data source = MYPC\SQLEXPRESS; database = MYDB; integrated security = TRUE";
conn.Open();
var query = "SELECT a.partNumber, b.fullPath,a.fileType,a.baseID FROM drawings a ,bases_bases b WHERE a.baseID = b.id";
using (var cmd = new SqlCommand(query, conn))
using (SqlDataReader dataReader = cmd.ExecuteReader())
{
while (dataReader.Read())
{
yield return new FileInfo()
{
partNumber = dataReader["partNumber"].ToString(),
path = dataReader["fullPath"].ToString(),
fileType = dataReader["fileType"].ToString(),
baseID = dataReader["baseID"].ToString(),
};
}
}
}
}
This will only create each FileInfo object as you iterate over the result returned by ReturnFileFromDBInfo:
foreach (var fileInfo in ReturnFileFromDBInfo())
{
string fileName = fileInfo.ToString();
// Create file etc.
}

The UserId, Password or Account is invalid Teradata .Net Connection

The below code throws an error
The UserId, Password or Account is invalid.
on the code line adapter.Fill(ds);
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using Teradata.Client.Provider;
using System.Data;
using System.Diagnostics;
public partial class _Default : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
TdConnectionStringBuilder connectionStringBuilder = new TdConnectionStringBuilder();
connectionStringBuilder.DataSource = "URL";
connectionStringBuilder.Database = "DB";
connectionStringBuilder.UserId = "USERNAME";
connectionStringBuilder.Password = "PASSWORD";
connectionStringBuilder.AuthenticationMechanism = "LDAP";
TdConnection cn = new TdConnection();
cn.ConnectionString = connectionStringBuilder.ConnectionString;
cn.Open();
TdCommand cmd = new TdCommand("EXEC MACRO", cn);
TdDataReader reader = cmd.ExecuteReader();
TdDataAdapter adapter = new TdDataAdapter(cmd.CommandText, cn.ConnectionString);
DataSet ds = new DataSet();
adapter.Fill(ds);
// myLabel.Text= ds.Tables[0].Rows[0]["event_id"].ToString();
cmd.Dispose();
cn.Close();
}
}
However, the below code works perfectly fine and returns value as expected.
TdConnectionStringBuilder connectionStringBuilder = new TdConnectionStringBuilder();
connectionStringBuilder.DataSource = "URL";
connectionStringBuilder.Database = "DB";
connectionStringBuilder.UserId = "USERNAME";
connectionStringBuilder.Password = "PASSWORD";
connectionStringBuilder.AuthenticationMechanism = "LDAP";
TdConnection cn = new TdConnection();
cn.ConnectionString = connectionStringBuilder.ConnectionString;
cn.Open();
TdCommand cmd = new TdCommand("Show table DB.TABLE1", cn);
String customers = (String)cmd.ExecuteScalar();
MeanTime.Text = customers;
cmd.Dispose();
cn.Close();
The user ID, Password, Datasource etc are all same yet it fails on 1st code but runs properly on 2nd.

When calling procs and macros you can use the TDCommand.ExecuteNonQuery.
UPDATE: Upon further reading here it looks like you can include the CALL when using the Stored Procedure commandtype. The proper Commandtype for a Macro execution is System.Data.CommandType.Text and you probably need the EXEC
For reference, with parameter binding and a procedure, here's a working example below. Some small tweaks would need to be made (as mentioned above) for a macro execution.
//create a new td connection
TdConnection cn = new TdConnection();
//Build the connection string
TdConnectionStringBuilder connectionStringBuilder = new TdConnectionStringBuilder();
connectionStringBuilder.DataSource = "serveraddress";
connectionStringBuilder.Database = "defaultdatabase";
connectionStringBuilder.UserId = "username";
connectionStringBuilder.Password = "password";
connectionStringBuilder.AuthenticationMechanism = "LDAP";
connectionStringBuilder.CommandTimeout = 120;
//Open the connection
cn.ConnectionString = connectionStringBuilder.ConnectionString;
cn.Open();
// Initialize TdCommand from the tdconnection
TdCommand cmd = cn.CreateCommand();
//CommandText is set to Stored Procedure name, in this case,
//.NET Data Provider will generate the CALL statement.
cmd.CommandText = "yourdatabasename.yourprocname";
cmd.CommandType = CommandType.StoredProcedure;
// Create Input Parameter required by this procedure
TdParameter InParm1 = cmd.CreateParameter();
InParm1.Direction = ParameterDirection.Input;
InParm1.DbType = DbType.String;
InParm1.Size = 20;
InParm1.Value = "yourparamvalue";
//and bind it
cmd.Parameters.Add(InParm1);
//------------OPTION 1 CATCHING AN OUTPUT PARAMETER---------------------
//If you are catching an output parameter from a proc then create here:
// Create Output Parameter.
TdParameter OutParm = cmd.CreateParameter();
OutParm.Direction = ParameterDirection.Output;
OutParm.ParameterName = "myOutputParam";
OutParm.DbType = DbType.String;
OutParm.Size = 200;
cmd.Parameters.Add(OutParm);
// Run it up
cmd.ExecuteReader()
//if this is returning a single value you can grab it now:
myOutput = OutParm.Value.ToString();
//------------OPTION 2 CATCHING A RECORDSET-----------------------------
//list based on class set in seperate model
List<myClass> l_myclass = new List<myClass>();
//run it up and catch into a TDDataRead
using (TdDataReader r = cmd.ExecuteReader())
{
if (r.HasRows)
{
//Loop the result set and catch the values in the list
while (r.Read())
{
//"myclass" class defined in a seperate model
//Obviously you could do whatever you want here, but
//creating a list on a class where the column1-column4 is defined makes short work of this.
//Then you can dump the whole l_myclass as json back to the client if you want.
myClass i = new myClass();
i.column1 = (!r.IsDBNull(0)) ? r.GetString(0) : string.Empty;
i.column2 = (!r.IsDBNull(1)) ? r.GetString(1) : string.Empty;
i.column3 = (!r.IsDBNull(2)) ? r.GetString(2) : string.Empty;
i.column4 = (!r.IsDBNull(3)) ? r.GetString(3) : string.Empty;
l_myClass.Add(i);
}
}
}
//Dump the list out as json (for example)
return Json(l_myClass, System.Web.Mvc.JsonRequestBehavior.AllowGet);
I've only, personally, used this on procedures, but the documentation I've read suggests that this is the correct route for macros as well.

How to insert and retrieve image from PostgreSql using C#

I am trying to insert an image into Postgres and retrieve that image from postgresql using C#.
I have two columns in my table: memberid (character varying) and member_photo (bytea)
Here is my code to insert the image:
using (FileStream pgFileStream = new FileStream("D:\\Capture.jpg", FileMode.Open, FileAccess.Read))
{
using (BinaryReader pgReader = new BinaryReader(new BufferedStream(pgFileStream)))
{
NpgsqlCommand command = new NpgsqlCommand();
byte[] ImgByteA = pgReader.ReadBytes(Convert.ToInt32(pgFileStream.Length));
command.CommandText = "insert into membermaster (memberid, member_photo) VALUES ('65', #Image)";
command.Connection = Program.conn;
Program.conn.Close();
Program.conn.Open();
//command.Parameters.Add(new NpgsqlParameter("Image", ImgByteA));
command.Parameters.Add("#Image", ImgByteA).Value = ImgByteA;
command.ExecuteNonQuery();
Program.conn.Close();
}
}
Here is my code to retrieve the image
NpgsqlCommand command = new NpgsqlCommand();
command.CommandText = "select member_photo from membermaster where memberid='65'";
command.Connection = Program.conn;
try
{
Program.conn.Close();
Program.conn.Open();
byte[] productImageByte = command.ExecuteScalar() as byte[];
if (productImageByte != null)
{
using (MemoryStream productImageStream = new System.IO.MemoryStream(productImageByte))
{
ImageConverter imageConverter = new System.Drawing.ImageConverter();
pictureBox1.Image = imageConverter.ConvertFrom(productImageByte) as System.Drawing.Image;
// image.Save(imageFullPath, System.Drawing.Imaging.ImageFormat.Jpeg);
pictureBox1.Image = System.Drawing.Image.FromStream(productImageStream);
}
}
}
catch
{
Program.conn.Close();
throw;
}
The code to insert the image is working fine, but I can't show this image in a picturebox.
I get an error :
Parameter is not valid
Please help me solve this problem

AFAIK you cannot retrieve a byte[] using ExecuteScalar. You should use ExecuteReader instead. Just to be on the safe side when inserting parameters, I prefer to specify types myself, so my insert looks like this:
using (var conn = new NpgsqlConnection(connString))
{
string sQL = "insert into picturetable (id, photo) VALUES(65, #Image)";
using (var command = new NpgsqlCommand(sQL, conn))
{
NpgsqlParameter param = command.CreateParameter();
param.ParameterName = "#Image";
param.NpgsqlDbType = NpgsqlTypes.NpgsqlDbType.Bytea;
param.Value = ImgByteA;
command.Parameters.Add(param);
conn.Open();
command.ExecuteNonQuery();
}
}
I can then retrieve and load the image like this:
using (var conn = new NpgsqlConnection(connString))
{
string sQL = "SELECT photo from picturetable WHERE id = 65";
using (var command = new NpgsqlCommand(sQL, conn))
{
byte[] productImageByte = null;
conn.Open();
var rdr = command.ExecuteReader();
if (rdr.Read())
{
productImageByte = (byte[])rdr[0];
}
rdr.Close();
if (productImageByte != null)
{
using (MemoryStream productImageStream = new System.IO.MemoryStream(productImageByte))
{
ImageConverter imageConverter = new System.Drawing.ImageConverter();
pictureBox1.Image = imageConverter.ConvertFrom(productImageByte) as System.Drawing.Image;
}
}
}
}
I do not know if specifying the datatype on insert makes any difference, so try just retrieving using a Reader first. If that doesn't work, then I suggest changing your insert routine to something like mine.
Please note in my example id is an integer, not a character varying!

While the code in the accepted answer is working, it is not true that you cannot use ExecuteScalar. It might have been an issue in older versions, but using Npgsql 6.0.7 the following works:
public static async Task<byte[]?> GetByteaAsync(int id, string connectionString) {
byte[]? result = null;
await using (var con = new NpgsqlConnection(connectionString)) {
await con.OpenAsync();
await using (var cmd = new NpgsqlCommand("SELECT field_name from table_name where id=#id", con) {CommandType = CommandType.Text}) {
cmd.Parameters.Add("#id", NpgsqlDbType.Integer).Value = id;
result = ((byte[]?) await cmd.ExecuteScalarAsync());
}
await con.CloseAsync();
}
return result;
}

Reading large text files into SQL Server stored procedure

I've got to read and put text into a SQL Server table.
I initially started with a BULK INSERT, but I have the problem that I don't now the encoding of the files and so the BULK sometimes outputs some errors.
Then, I choose to make through a C# Console Aplication, and everything works fine, except when I have large files. To about 300-400 MB, I don't have any problem, but above that, I'm getting a OOM error.
Here's my code:
static void Main()
{
string fullFileName = #"C:\Temp\410.4604";
string FileName = System.IO.Path.GetFileName(fullFileName);
string table = FileName.Substring(0, 3);
Console.WriteLine("Iniciou às: " + inicio);
DataTable data = new DataTable();
data.Clear();
data.Columns.Add("Text");
using (System.IO.FileStream fs = System.IO.File.Open(fullFileName, System.IO.FileMode.Open, System.IO.FileAccess.Read, System.IO.FileShare.Read))
using (System.IO.BufferedStream bs = new System.IO.BufferedStream(fs))
using (StreamReader sr = new StreamReader(bs))
{
string line;
while ((line = sr.ReadLine()) != null)
{
data.Rows.Add(line);
}
}
var conn = new System.Data.SqlClient.SqlConnection(myConn);
SqlCommand insertCommand = new SqlCommand(myProc, conn);
insertCommand.CommandTimeout = 0;
insertCommand.CommandType = CommandType.StoredProcedure;
SqlParameter insParam1 = insertCommand.Parameters.AddWithValue("#data", data);
insParam1.SqlDbType = SqlDbType.Structured;
SqlParameter insParam2 = insertCommand.Parameters.AddWithValue("#table", table);
conn.Open();
insertCommand.ExecuteNonQuery();
conn.Close();
data.Clear();
data = null;
GC.Collect();
Console.WriteLine("Press any key to exit.");
System.Console.ReadKey();
}
Any ideas? Or advices to improve this?
Many thanks

Can't put varbinary from SQL into a byte[] in C# program using SqlDataReader and ExecuteReader

I have seen many solutions to this problem where people will just use Command.ExecuteScalar as byte[]; but their SQL queries are getting one varbinary field at a time. I am trying to select about 30k rows of varbinary entries, but them in a byte[] and deserialize.
Here is my code:
public void MNAdapter()
{
IsoStorage retVal = new IsoStorage();
SqlConnectionStringBuilder csb = new SqlConnectionStringBuilder();
csb.DataSource = #"LocalMachine\SQLDEV";
csb.InitialCatalog = "Support";
csb.IntegratedSecurity = true;
string connString = csb.ToString();
using (SqlConnection conn = new SqlConnection(connString))
{
conn.Open();
SqlCommand command = conn.CreateCommand();
command.CommandText = #"SELECT S.Settings
from Support.dbo.SavedLocalSettings S
inner join WebCatalog.Published.People P
on P.PKey = S.PeopleLink
inner join WebCatalog.Published.Company C
on P.Link = C.PeopleList
where S.DateSaved >= GETDATE()-34
and C.PKey != '530F4622-C30D-DD11-A23A-00304834A8C9'
and C.PKey != '7BAF7229-9249-449E-BEA5-4B366D7ECCD1'
and C.PKey != 'CCBB2140-C30D-DD11-A23A-00304834A8C9'
and S.CompanyName not like 'Tech Support%'
Group By S.PeopleLink, S.Settings";
using (SqlDataReader reader = command.ExecuteReader(CommandBehavior.SequentialAccess))
{
//DataTable dt = new DataTable();
//dt.Load(reader);
byte[] blob = null;
BinaryFormatter bf = new BinaryFormatter();
bf.Binder = new CustomBinder();
while (reader.Read())
{
reader.GetBytes(0,0,blob,0,100000);
Console.WriteLine(blob.ToString());
retVal = bf.Deserialize(new MemoryStream(blob)) as IsoStorage;
}
}
}
}
I also tried putting them in a Data tabla first even though I thought that would be redundant, but they get read in as integers.
I don't get any errors and the data is going into the data reader, but it's like reader.GetBytes(0,0,blob,0,100000); is not even running because blob stays null.

Why not use:
blob = (byte[])reader.Items["Settings"];
Or
blob = (byte[])reader["Settings"];

reader.GetBytes(0,0,blob,0,100000);
You expect this method to create the array of bytes for you. It won't - it needs a reference to an existing array. You have to prepare the array yourself.